H MySQL and the future (The TODO)

This appendix lists the features that we plan to implement in MySQL.

Everything in this list is approximately in the order it will be done. If you want to affect the priority order, please register a license or support us and tell us what you want to have done more quickly. See section 3 MySQL Licensing and Support.

The plan is that we in the future will support the full ANSI SQL99 standard, but with a lot of useful extensions. The challenge is to do this without sacrifying the speed or compromise the code.

H.1 Things that should be in 4.0

We plan to make MySQL Version 4.0 a ``quick'' release where we only add some new stuff to enable others to help us with developing new features into Version 4.1. The MySQL 4.0 version should only take us about a month to make after which we want to stabilize it and start working on Version 4.1. Version 4.0 should have the following new features:

The news section for 4.0 includes a list of the features we have already implemented in the 4.0 tree. See section F.1 Changes in release 4.0.x (Development; Alpha).

• New table definition file format (.frm files) This will enable us to not run out of bits when adding more table options. One will still be able to use the old .frm file format with 4.0. All newly created tables will, however, use the new format. The new file format will enable us to add new column types, more options for keys and FOREIGN KEY support.
• mysqld as a library. This will have the same interface as the standard MySQL client (with an extra function to just set up startup parameters) but will be faster (no TCP/IP or socket overhead), smaller and much easier to use for embedded products. One will be able to define at link time if one wants to use the client/server model or a stand-alone application just by defining which library to link with. The mysqld will support all standard MySQL features and one can use it in a threaded client to run different queries in each thread.
• Replication should work with RAND().
• Online backup with very low performance penalty. The online backup will make it easy to add a new replication slave without taking down the master.
• DELETE FROM table_name will return the number of deleted rows. For fast execution one should use TRUNCATE table_name.
• Multi-table DELETE (cascading DELETE and multi-table DELETE.
• Allow DELETE on MyISAM tables to use the record cache. To do this, we need to update the threads record cache when we update the .MYD file.
• Better replication.
• More functions for full-text search. See section 12.2 New Features of Full-text Search to Appear in MySQL 4.0.
• Character set casts and syntax for handling multiple character sets.
• Allow users to change startup options without taking down the server.
• Help for all commands from the client.
• Secure connections (with SSL).
• Extend the optimizer to be able to optimize some ORDER BY key_name DESC queries.
• SHOW COLUMNS FROM table_name (used by mysql client to allow expansions of column names) should not open the table, but only the definition file. This will require less memory and be much faster.
• New key cache
• When using SET CHARACTER SET we should translate the whole query at once and not only strings. This will enable users to use the translated characters in database, table and column names.
• Add a portable interface over gethostbyaddr_r() so that we can change ip_to_hostname() to not block other threads while doing DNS lookups.
• Add record_in_range() method to MERGE tables to be able to choose the right index when there is many to choose from. We should also extend the info interface to get the key distribution for each index, of analyze is run on all sub tables.
• SET SQL_DEFAULT_TABLE_TYPE=[MyISAM | INNODB | BDB | GEMINI | HEAP].

H.2 Things that must be done in the real near future

• Fail safe replication.
• Subqueries. select id from t where grp in (select grp from g where u > 100)
• Derived tables.

  select a.col1, b.col2 from (select max(col1) as col1 from root_table ) a,
  other_table b where a.col1=b.col1
  

  This could be done by automatically creating temporary tables for the derived tables for the duration of the query.
• Add PREPARE of statements and sending of parameters to mysqld.
• Extend the server/client protocol to support warnings.
• Add options to the server/protocol protocol to get progress notes for long running commands.
• Add database and real table name (in case of alias) to the MYSQL_FIELD structure.
• Don't allow more than a defined number of threads to run MyISAM recover at the same time.
• Change INSERT ... SELECT to use concurrent inserts.
• Return the original field types() when doing SELECT MIN(column) ... GROUP BY.
• Multiple result sets.
• Change the protocol to allow binary transfer of values. To do this efficiently, we need to add an API to allow binding of variables.
• Make it possible to specify long_query_time with a granularity in microseconds.
• Add range checking to MERGE tables.
• Port of MySQL to BeOS.
• Link the myisampack code into the server.
• Add a temporary key buffer cache during INSERT/DELETE/UPDATE so that we can gracefully recover if the index file gets full.
• If you perform an ALTER TABLE on a table that is symlinked to another disk, create temporary tables on this disk.
• Implement a DATE/DATETIME type that handles time zone information properly, so that dealing with dates in different time zones is easier.
• FreeBSD and MIT-pthreads; Do sleeping threads take CPU?
• Check if locked threads take any CPU.
• Fix configure so that one can compile all libraries (like MyISAM) without threads.
• Add an option to periodically flush key pages for tables with delayed keys if they haven't been used in a while.
• Allow join on key parts (optimization issue).
• INSERT SQL_CONCURRENT and mysqld --concurrent-insert to do a concurrent insert at the end of the file if the file is read-locked.
• Remember FOREIGN key definitions in the `.frm' file.
• Server side cursors.
• Check if lockd works with modern Linux kernels; If not, we have to fix lockd! To test this, start mysqld with --enable-locking and run the different fork* test suits. They shouldn't give any errors if lockd works.
• Allow SQL variables in LIMIT, like in LIMIT @a,@b.
• Don't add automatic DEFAULT values to columns. Give an error when using an INSERT that doesn't contain a column that doesn't have a DEFAULT.
• Caching of queries and results. This should be done as a separated module that examines each query and if this is query is in the cache the cached result should be returned. When one updates a table one should remove as few queries as possible from the cache. This should give a big speed bost on machines with much RAM where queries are often repeated (like WWW applications). One idea would be to only cache queries of type: SELECT CACHED ...
• Fix `libmysql.c' to allow two mysql_query() commands in a row without reading results or give a nice error message when one does this.
• Optimize BIT type to take 1 bit (now BIT takes 1 char).
• Check why MIT-pthreads ctime() doesn't work on some FreeBSD systems.
• Add an IMAGE option to LOAD DATA INFILE to not update TIMESTAMP and AUTO_INCREMENT fields.
• Added LOAD DATE INFILE.. UPDATE syntax.
  • For tables with primary keys, if the data contains the primary key, entries matching that primary key are updated from the remainder of the columns. However, columns MISSING from the incoming data feed are not touched.
  • For tables tables with primary keys that are missing some part of the key in the incoming data stream, or that have no primary key, the feed is treated as a LOAD DATA INFILE ... REPLACE INTO is now.
• Make LOAD DATA INFILE understand syntax like:

  LOAD DATA INFILE 'file_name.txt' INTO TABLE tbl_name
  TEXT_FIELDS (text_field1, text_field2, text_field3)
  SET table_field1=concatenate(text_field1, text_field2), table_field3=23
  IGNORE text_field3
  
  This can be used to skip over extra columns in the text file, or update columns
  based on expressions of the read data...
  

• LOAD DATA INFILE 'file_name' INTO TABLE 'table_name' ERRORS TO err_table_name This would cause any errors and warnings to be logged into the err_table_name table. That table would have a structure like:

  line_number    - line number in data file
  error_message  - the error/warning message
  and maybe
  data_line      - the line from the data file
  

• Add true VARCHAR support (There is already support for this in MyISAM).
• Automatic output from mysql to netscape.
• LOCK DATABASES. (with various options)
• Change sort to allocate memory in ``hunks'' to get better memory utilization.
• DECIMAL and NUMERIC types can't read exponential numbers; Field_decimal::store(const char *from,uint len) must be recoded to fix this.
• Fix mysql.cc to do fewer malloc() calls when hashing field names.
• Functions: ADD_TO_SET(value,set) and REMOVE_FROM_SET(value,set)
• Add use of t1 JOIN t2 ON ... and t1 JOIN t2 USING ... Currently, you can only use this syntax with LEFT JOIN.
• Add full support for unsigned long long type.
• Many more variables for show status. Counts for: INSERT/DELETE/UPDATE statements. Records reads and updated. Selects on 1 table and selects with joins. Mean number of tables in select. Number of ORDER BY and GROUP BY queries.
• If you abort mysql in the middle of a query, you should open another connection and kill the old running query. Alternatively, an attempt should be made to detect this in the server.
• Add a handler interface for table information so you can use it as a system table. This would be a bit slow if you requested information about all tables, but very flexible. SHOW INFO FROM tbl_name for basic table information should be implemented.
• Add support for UNICODE.
• NATURAL JOIN and UNION JOIN
• Allow select a from crash_me left join crash_me2 using (a); In this case a is assumed to come from the crash_me table.
• Fix that ON and USING works with the JOIN and INNER JOIN join types.
• Oracle like CONNECT BY PRIOR ... to search hierarchy structures.
• RENAME DATABASE
• mysqladmin copy database new-database. -- Requires COPY command to be added to mysqld
• Processlist should show number of queries/thread.
• SHOW HOSTS for printing information about the hostname cache.
• DELETE and REPLACE options to the UPDATE statement (this will delete rows when one gets a duplicate key error while updating).
• Change the format of DATETIME to store fractions of seconds.
• Add all missing ANSI92 and ODBC 3.0 types.
• Change table names from empty strings to NULL for calculated columns.
• Don't use 'Item_copy_string' on numerical values to avoid number->string->number conversion in case of: SELECT COUNT(*)*(id+0) FROM table_name GROUP BY id
• Make it possible to use the new GNU regexp library instead of the current one (The GNU library should be much faster than the old one).
• Change that ALTER TABLE doesn't abort clients that executes INSERT DELAYED.
• Fix that when columns referenced in an UPDATE clause contains the old values before the update started.
• Allow update of variables in UPDATE statements. For example: UPDATE TABLE foo SET @a=a+b,a=@a, b=@a+c
• myisamchk, REPAIR and OPTIMIZE TABLE should be able to handle cases where the data and/or index files are symbolic links.
• Add simulation of pread()/pwrite() on Windows to enable concurrent inserts.
• A logfile analyzer that could parsed out information about which tables are hit most often, how often multi-table joins are executed, etc. It should help users identify areas or table design that could be optimized to execute much more efficient queries.
• Add SUM(DISTINCT)
• Add ANY(),EVERY() and SOME() group functions. In ANSI SQL these only works on boolean columns, but we can extend these to work on any columns/expressions by applying: value == 0 -> FALSE and value <> 0 -> TRUE.
• Fix that the type for MAX(column) is the same as the column type.

  create table t1 (a DATE);
  insert into t1 values (now());
  create table t2 select max(a) from t1;
  show columns from t2;
  

• Come up with a nice syntax for a statement that will UPDATE the row if it exists and INSERT a new row if the row didn't exist. (Like REPLACE works with INSERT / DELETE)

H.3 Things that have to be done sometime

• Implement function: get_changed_tables(timeout,table1,table2,...)
• Atomic multi-table updates, eg update items,month set items.price=month.price where items.id=month.id;;
• Change reading through tables to use memmap when possible. Now only compressed tables use memmap.
• Add a new privilege 'Show_priv' for SHOW commands.
• Make the automatic timestamp code nicer. Add timestamps to the update log with SET TIMESTAMP=#;
• Use read/write mutex in some places to get more speed.
• Full foreign key support. One probably wants to implement a procedural language first.
• Simple views (first on one table, later on any expression).
• Automatically close some tables if a table, temporary table or temporary files gets error 23 (not enough open files).
• When one finds a field=#, change all occurrences of field to #. Now this is only done for some simple cases.
• Change all const expressions with calculated expressions if possible.
• Optimize key = expression. At the moment only key = field or key = constant are optimized.
• Join some of the copy functions for nicer code.
• Change `sql_yacc.yy' to an inline parser to reduce its size and get better error messages (5 days).
• Change the parser to use only one rule per different number of arguments in function.
• Use of full calculation names in the order part. (For ACCESS97)
• UNION, MINUS, INTERSECT and FULL OUTER JOIN. (Currently only LEFT OUTER JOIN is supported)
• Allow UNIQUE on fields that can be NULL.
• SQL_OPTION MAX_SELECT_TIME=# to put a time limit on a query.
• Make the update log to a database.
• Negative LIMIT to retrieve data from the end.
• Alarm around client connect/read/write functions.
• Please note the changes to safe_mysqld: according to FSSTND (which Debian tries to follow) PID files should go into `/var/run/<progname>.pid' and log files into `/var/log'. It would be nice if you could put the "DATADIR" in the first declaration of "pidfile" and "log", so the placement of these files can be changed with a single statement.
• Allow a client to request logging.
• Add use of zlib() for gzip-ed files to LOAD DATA INFILE.
• Fix sorting and grouping of BLOB columns (partly solved now).
• Stored procedures. This is currently not regarded to be very important as stored procedures are not very standardized yet. Another problem is that true stored procedures make it much harder for the optimizer and in many cases the result is slower than before We will, on the other hand, add a simple (atomic) update language that can be used to write loops and such in the MySQL server.
• Change to use semaphores when counting threads. One should first implement a semaphore library to MIT-pthreads.
• Don't assign a new AUTO_INCREMENT value when one sets a column to 0. Use NULL instead.
• Add full support for JOIN with parentheses.
• As an alternative for one thread / connection manage a pool of threads to handle the queries.
• Allow one to get more than one lock with GET_LOCK. When doing this, one must also handle the possible deadlocks this change will introduce.

Time is given according to amount of work, not real time.

H.4 Some things we don't have any plans to do

• Nothing; In the long run we plan to be fully ANSI 92 / ANSI 99 compliant.

Go to the first, previous, next, last section, table of contents.