Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
513 changes: 513 additions & 0 deletions mysql-test/suite/innodb/r/ext_buf_pool.result

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
SELECT * FROM INFORMATION_SCHEMA.INNODB_BUFFER_POOL_STATS;
POOL_ID POOL_SIZE FREE_BUFFERS DATABASE_PAGES OLD_DATABASE_PAGES MODIFIED_DATABASE_PAGES PENDING_DECOMPRESS PENDING_READS PENDING_FLUSH_LRU PENDING_FLUSH_LIST PAGES_MADE_YOUNG PAGES_NOT_MADE_YOUNG PAGES_MADE_YOUNG_RATE PAGES_MADE_NOT_YOUNG_RATE NUMBER_PAGES_READ NUMBER_PAGES_CREATED NUMBER_PAGES_WRITTEN PAGES_READ_RATE PAGES_CREATE_RATE PAGES_WRITTEN_RATE NUMBER_PAGES_GET HIT_RATE YOUNG_MAKE_PER_THOUSAND_GETS NOT_YOUNG_MAKE_PER_THOUSAND_GETS NUMBER_PAGES_READ_AHEAD NUMBER_READ_AHEAD_EVICTED READ_AHEAD_RATE READ_AHEAD_EVICTED_RATE LRU_IO_TOTAL LRU_IO_CURRENT UNCOMPRESS_TOTAL UNCOMPRESS_CURRENT
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
POOL_ID POOL_SIZE FREE_BUFFERS DATABASE_PAGES OLD_DATABASE_PAGES MODIFIED_DATABASE_PAGES PENDING_DECOMPRESS PENDING_READS PENDING_FLUSH_LRU PENDING_FLUSH_LIST PAGES_MADE_YOUNG PAGES_NOT_MADE_YOUNG PAGES_MADE_YOUNG_RATE PAGES_MADE_NOT_YOUNG_RATE NUMBER_PAGES_READ NUMBER_PAGES_CREATED NUMBER_PAGES_WRITTEN PAGES_READ_RATE PAGES_CREATE_RATE PAGES_WRITTEN_RATE NUMBER_PAGES_GET HIT_RATE YOUNG_MAKE_PER_THOUSAND_GETS NOT_YOUNG_MAKE_PER_THOUSAND_GETS NUMBER_PAGES_READ_AHEAD NUMBER_READ_AHEAD_EVICTED READ_AHEAD_RATE READ_AHEAD_EVICTED_RATE LRU_IO_TOTAL LRU_IO_CURRENT UNCOMPRESS_TOTAL UNCOMPRESS_CURRENT NUMBER_PAGES_WRITTEN_TO_EXTERNAL_BUFFER_POOL NUMBER_PAGES_READ_FROM_EXTERNAL_BUFFER_POOL
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
CREATE TABLE infoschema_buffer_test (col1 INT) ENGINE = INNODB;
INSERT INTO infoschema_buffer_test VALUES(9);
SELECT * FROM INFORMATION_SCHEMA.INNODB_BUFFER_PAGE
Expand Down
2 changes: 2 additions & 0 deletions mysql-test/suite/innodb/r/innodb_status_variables.result
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,10 @@ INNODB_BUFFER_POOL_READ_AHEAD
INNODB_BUFFER_POOL_READ_AHEAD_EVICTED
INNODB_BUFFER_POOL_READ_REQUESTS
INNODB_BUFFER_POOL_READS
INNODB_EXT_BUFFER_POOL_READS
INNODB_BUFFER_POOL_WAIT_FREE
INNODB_BUFFER_POOL_WRITE_REQUESTS
INNODB_EXT_BUFFER_POOL_PAGES_FLUSHED
INNODB_CHECKPOINT_AGE
INNODB_CHECKPOINT_MAX_AGE
INNODB_DATA_FSYNCS
Expand Down
1 change: 1 addition & 0 deletions mysql-test/suite/innodb/t/ext_buf_pool.opt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
--innodb-buffer-pool-size=21M --innodb-extended-buffer-pool-size=1M
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typically, the local SSD is larger than the RAM, and therefore it would be more interesting to test a scenario where the buffer pool is smaller than the extended buffer pool. I understand that our current CI setup is not suitable for any performance testing. But, currently we only cover this functionality in debug-instrumented executables.

Would it be possible to write some sort of a test, or extend an existing one (such as encryption.innochecksum) with a variant that would use the minimal innodb_buffer_pool_size and a larger innodb_extended_buffer_pool_size? Currently, that test is running with innodb_buffer_pool_size=64M.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test flushes small amount of pages into extended buffer pool file, that's why there was not any reason to set a big file size. Probably it makes sense to develop separate test, we can set the minimal innodb_buffer_pool_size and a larger innodb_extended_buffer_pool_size, but what exactly are we going to test in that separate test?

169 changes: 169 additions & 0 deletions mysql-test/suite/innodb/t/ext_buf_pool.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
--source include/have_innodb.inc
--source include/have_debug.inc
--source include/have_debug_sync.inc
Comment on lines +1 to +3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we have a non-debug version of this test as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we can't. We have to inject debug code to have ability to test it.

--source include/count_sessions.inc
--source ../encryption/include/have_file_key_management_plugin.inc
#--source include/innodb_page_size.inc

--let $encrypted_row_compressed=6
--let $unencrypted_row_compressed=5
--let $unencrypted_uncompressed=4
--let $encrypted_uncompressed=3
--let $unencrypted_page_compressed=2
--let $encrypted_page_compressed=1
--let $i = $encrypted_row_compressed

--let $page_size=`SELECT @@GLOBAL.innodb_page_size`
if ($page_size != 16384) {
--let $i=$unencrypted_uncompressed
}

--connect (prevent_purge,localhost,root)
START TRANSACTION WITH CONSISTENT SNAPSHOT;

--connection default

--let $DATADIR = `select @@datadir`
# Set ext buffer pool file size and check it was created, check it's size
--list_files $DATADIR ext_buffer_pool

--disable_query_log
--error 0,ER_UNKNOWN_SYSTEM_VARIABLE
SET @old_innodb_limit_optimistic_insert_debug = @@innodb_limit_optimistic_insert_debug;
SET @old_debug_dbug = @@debug_dbug;
--enable_query_log

--error 0,ER_UNKNOWN_SYSTEM_VARIABLE
SET GLOBAL innodb_limit_optimistic_insert_debug = 3;
SET GLOBAL DEBUG_DBUG='+d,ib_ext_bp_count_io_only_for_t';

while($i) {

SET GLOBAL DEBUG_DBUG='+d,ib_ext_bp_disable_LRU_eviction_for_t';
if ($i == $unencrypted_uncompressed) {
--echo ###################################################################
--echo # Testing for unencrypted uncompressed table #
--echo ###################################################################
CREATE TABLE t (
`a` INT NOT NULL,
PRIMARY KEY (`a`)
) ENGINE=InnoDB STATS_PERSISTENT=0;
}
if ($i == $encrypted_uncompressed) {
--echo ###################################################################
--echo # Testing for encrypted uncompressed table #
--echo ###################################################################
CREATE TABLE t (
`a` INT NOT NULL,
PRIMARY KEY (`a`)
) ENGINE=InnoDB STATS_PERSISTENT=0 encrypted=yes encryption_key_id=1;
}
if ($i == $unencrypted_page_compressed) {
--echo ###################################################################
--echo # Testing for unencrypted PAGE_COMPRESSED=1 table #
--echo ###################################################################
CREATE TABLE t (
`a` INT NOT NULL,
PRIMARY KEY (`a`)
) ENGINE=InnoDB STATS_PERSISTENT=0 PAGE_COMPRESSED=1;
}
if ($i == $unencrypted_row_compressed) {
--echo ###################################################################
--echo # Testing for unencrypted ROW_FORMAT=COMPRESSED table #
--echo ###################################################################
CREATE TABLE t (
`a` INT NOT NULL,
PRIMARY KEY (`a`)
) ENGINE=InnoDB STATS_PERSISTENT=0 ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=1;
}
if ($i == $encrypted_page_compressed) {
--echo ###################################################################
--echo # Testing for encrypted PAGE_COMPRESSED=1 table #
--echo ###################################################################
CREATE TABLE t (
`a` INT NOT NULL,
PRIMARY KEY (`a`)
) ENGINE=InnoDB STATS_PERSISTENT=0 PAGE_COMPRESSED=1 encrypted=yes
encryption_key_id=1;
}
if ($i == $encrypted_row_compressed) {
--echo ###################################################################
--echo # Testing for encrypted ROW_FORMAT=COMPRESSED table #
--echo ###################################################################
CREATE TABLE t (
`a` INT NOT NULL,
PRIMARY KEY (`a`)
) ENGINE=InnoDB STATS_PERSISTENT=0 ROW_FORMAT=COMPRESSED encrypted=yes
encryption_key_id=1;
}

SELECT variable_value INTO @prev_flushed_gs
FROM information_schema.global_status
WHERE variable_name LIKE 'INNODB_EXT_BUFFER_POOL_PAGES_FLUSHED';
SELECT NUMBER_PAGES_WRITTEN_TO_EXTERNAL_BUFFER_POOL INTO @prev_written_ps
FROM INFORMATION_SCHEMA.INNODB_BUFFER_POOL_STATS;
SELECT variable_value INTO @prev_reads_gs
FROM information_schema.global_status
WHERE variable_name LIKE 'INNODB_EXT_BUFFER_POOL_READS';
SELECT NUMBER_PAGES_READ_FROM_EXTERNAL_BUFFER_POOL INTO @prev_reads_ps
FROM INFORMATION_SCHEMA.INNODB_BUFFER_POOL_STATS;

--eval SET @start_val = $i*100
INSERT INTO t SET a = @start_val+1;
INSERT INTO t SET a = @start_val+2;
INSERT INTO t SET a = @start_val+3;
INSERT INTO t SET a = @start_val+4;
INSERT INTO t SET a = @start_val+5;
INSERT INTO t SET a = @start_val+6;
INSERT INTO t SET a = @start_val+7;
INSERT INTO t SET a = @start_val+8;
INSERT INTO t SET a = @start_val+9;
INSERT INTO t SET a = @start_val+10;
INSERT INTO t SET a = @start_val+11;
INSERT INTO t SET a = @start_val+12;

SET GLOBAL DEBUG_DBUG='-d,ib_ext_bp_disable_LRU_eviction_for_t';
SET GLOBAL innodb_force_LRU_eviction = TRUE;

let $wait_condition =
SELECT (variable_value-@prev_flushed_gs) >= 9
FROM information_schema.global_status
WHERE variable_name LIKE 'INNODB_EXT_BUFFER_POOL_PAGES_FLUSHED';
--source include/wait_condition.inc

SELECT variable_value-@prev_flushed_gs
FROM information_schema.global_status
WHERE variable_name LIKE 'INNODB_EXT_BUFFER_POOL_PAGES_FLUSHED';
SELECT NUMBER_PAGES_WRITTEN_TO_EXTERNAL_BUFFER_POOL-@prev_written_ps
FROM INFORMATION_SCHEMA.INNODB_BUFFER_POOL_STATS;
SELECT variable_value-@prev_reads_gs
FROM information_schema.global_status
WHERE variable_name LIKE 'INNODB_EXT_BUFFER_POOL_READS';
SELECT NUMBER_PAGES_READ_FROM_EXTERNAL_BUFFER_POOL-@prev_reads_ps
FROM INFORMATION_SCHEMA.INNODB_BUFFER_POOL_STATS;

SELECT * FROM t;

SELECT variable_value-@prev_flushed_gs
FROM information_schema.global_status
WHERE variable_name LIKE 'INNODB_EXT_BUFFER_POOL_PAGES_FLUSHED';
SELECT NUMBER_PAGES_WRITTEN_TO_EXTERNAL_BUFFER_POOL-@prev_written_ps
FROM INFORMATION_SCHEMA.INNODB_BUFFER_POOL_STATS;
SELECT variable_value-@prev_reads_gs
FROM information_schema.global_status
WHERE variable_name LIKE 'INNODB_EXT_BUFFER_POOL_READS';
SELECT NUMBER_PAGES_READ_FROM_EXTERNAL_BUFFER_POOL-@prev_reads_ps
FROM INFORMATION_SCHEMA.INNODB_BUFFER_POOL_STATS;

DROP TABLE t;
--dec $i
}

--disable_query_log
SET GLOBAL DEBUG_DBUG=@old_debug_dbug;
--error 0,ER_UNKNOWN_SYSTEM_VARIABLE
SET GLOBAL innodb_limit_optimistic_insert_debug = @old_innodb_limit_optimistic_insert_debug;
--enable_query_log

--disconnect prevent_purge
--source include/wait_until_count_sessions.inc
4 changes: 3 additions & 1 deletion mysql-test/suite/innodb_i_s/innodb_buffer_pool_stats.result
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,7 @@ INNODB_BUFFER_POOL_STATS CREATE TEMPORARY TABLE `INNODB_BUFFER_POOL_STATS` (
`LRU_IO_TOTAL` bigint(21) unsigned NOT NULL,
`LRU_IO_CURRENT` bigint(21) unsigned NOT NULL,
`UNCOMPRESS_TOTAL` bigint(21) unsigned NOT NULL,
`UNCOMPRESS_CURRENT` bigint(21) unsigned NOT NULL
`UNCOMPRESS_CURRENT` bigint(21) unsigned NOT NULL,
`NUMBER_PAGES_WRITTEN_TO_EXTERNAL_BUFFER_POOL` bigint(21) unsigned NOT NULL,
`NUMBER_PAGES_READ_FROM_EXTERNAL_BUFFER_POOL` bigint(21) unsigned NOT NULL
Comment on lines -35 to +37
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test rocksdb.innodb_i_s_tables_disabled needs to be adjusted as well.

) ENGINE=MEMORY DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_general_ci
Loading
Loading