Announcing MySQL Utilities release-1.5.2!

The MySQL Utilities Team is pleased to announce the general availability (GA) release of MySQL Utilities. This release includes a number of improvements for useabilty, stability, and a few enhancements. A complete list of all improvements can be found in our release_notes.

New Enhancements!

This release represents a stable release of the product. Along with several defect patches, we also include the following enhancements.

• The mysqlserverclone utility now checks diskspace as a prerequisite to cloning the server.
• The --force option was removed from the mysqlfailover utility to remove confusion on its purpose.
• A warning is issued if the mysqld --skip-innodb option is used because this option is ignored in MySQL 5.7 and later.
• Utilities can correctly parse and display the PROXY privilege for GRANT statements.

 

How Can I Download MySQL Utilities?

You can download MySQL Utilities 1.5.2 GA from the following link using one of the pre-built installation repositories including a source download. http://dev.mysql.com/downloads/tools/utilities/

Where is the Documentation?

You can find online documentation for MySQL Utilities version 1.5 at: http://dev.mysql.com/doc/index-utils-fabric.html

Announcing MySQL Utilities release-1.6.0-alpha

The MySQL Utilities Team is pleased to announce the alpha release of MySQL Utilities. This release represents a new evolution of the product including the following enhancements.

• The mysqlprocgrep utility now displays the processes and connections killed during a --kill operation. The displayed rows come from the appropriate SHOW PROCESSLIST entries.
• The mysqlbinlogmove utility was added. It relocates binary log files, and moves files based on their sequence number or modified date.
• The mysqlgrants utility was added. It displays grants per object, and produces reports by user, user with grants, and GRANT statement.
• Health reports can now be generated for a list of slaves without a master specified. In this case, "No master specified" appears for connection status for each slave, instead of an error.

 

How Can I Download MySQL Utilities?

You can download MySQL Utilities 1.6.0 Alpha from the following link using one of the pre-built installation repositories including a source download. Click on the Development Releases tab. http://dev.mysql.com/downloads/tools/utilities/

Do you want to learn more about DevOps and MySQL?

Are you attending Oracle Open World? Would you like to know more about Oracle DevOps products for managing your MySQL servers? If so, stop by Tuesday, September 30 at 10:15 in the Hotel Nikko - Bay View and join us for a hands on lab (HOL9234) for MySQL Utilities and Connector/Python.

We will present the basics for getting started using Python to manage your MySQL servers. MySQL Utilities includes utilities for a wide variety of operations from displaying GRANT statements to setting up automatic failover for replication.

Check out the abstract for the HOL here.

3D Printing Mastery Just Got a Little Easier

My new book on 3D printing has been published! I must say this work has been a real joy to write. It's the book I wish existed when I first started playing with 3D printers.

Like most people who started with building their own 3D printer from scratch, I struggled to learn the concepts and get a firm foundation for how the machines work and how to work with them. I was dismayed by the lack of documentation and help available from most vendors. Indeed, most tend to say things like "do a Google search" rather than answer your questions directly.

Fortunately, there is a lot of information out there about 3D printing, especially from the RepRap community. Unfortunately, the sheer volume can be overwhelming for most and in some cases the information is either not entirely correct or is too specific for a certain printer to be a solution you can use. Furthermore, it can take hours to sift through the data to find what you need and even more time to separate fact from fiction.

I've already been through that and spent years sifting through the data. I have condensed everything I've learned into a medium-sized book that I think you will find to be the missing link from opening the box and setting up your printer for the first time to printing quality prints with confidence.

I cover topics ranging from how to calibrate your printer to troubleshooting hardware, software, and even printing problems. While it won't make you a master of your 3D printer overnight, it will take you a lot further along in a short amount of time!

Check it out: http://www.apress.com/9781430268093

New! MySQL Utilities release-1.4.2-RC

The MySQL Utilities Team is pleased to announce the latest release candidate (RC) release of MySQL Utilities. This release includes a number of improvements for useabilty, stability, and a few enhancements. A complete list of all improvements can be found in our release_notes.

New Utilities!

We have also included two new utilities.

• The mysqlrplsync utility was added, which checks data consistency between servers in a replicated setup. 
• The mysqlrplms utility was added, which provides round-robin multi-source replication (a slave server continually cycles through multiple masters in order to store a consolidated data set).

How Can I Download MySQL Utilities?

You can download MySQL Utilities 1.4.2 from the following link using one of the pre-built installation repositories including a source download. Click on the Development Releases tab.

http://dev.mysql.com/downloads/tools/utilities/

Where is the Documentation?

You can find online documentation for MySQL Utilities version 1.4 at:

http://dev.mysql.com/doc/index-gui.html

Announcing MySQL Connector/Arduino 1.0.2 Beta

I've completed a new release of the Connector/Arduino! The new release contains some major improvements with memory handling.

• The library has been trimmed to save memory.
• Static strings moved to PROGMEM strings
• Unused structures removed (e.g. ok_packet)
• Moved two more methods to optional compilation
• The WITH_SELECT is turned *OFF* by default. If you want to use select queries, be sure to uncomment this in the mysql.h file.
• Added a CHANGES.txt file to track changes between releases.

Memory, What Memory?

If you have used previous versions of the connector in medium to large sketches or have long query strings or even many variables, chances are you have hit the memory limit for your wee Arduino board.

This can manifest itself in a number of ways. Most notably, the sketch may work for you until you add more code or more sensors in which it can fail to connect to the server. It can also exhibit random reboots or random freezes. If this is happening to you, it is most likely a memory issue.

That is, the old version of the connector consumed nearly 70% of available dynamic memory - the memory used for variables. When the Arduino exceeds its memory limit, sketches will exhibit strange behavior or the board may reboot.

Many people have encountered this so I've worked hard to try and squeeze more memory out of the connector. Which isn't easy considering it must keep a buffer of the data being sent to (or received from) the server.

Let's see an example. The old version of the connector (1.0.1b), when compiled with the hello_mysql example for a Uno, consumes about 68% of available dynamic memory leaving only 637 bytes for your own variables. That's fine for a simple sketch but if you want to do complex queries building INSERT statements from several sensors or including other libraries for additional features (like an LCD), you're not going to be happy.

While you can (and should) limit your memory use and even make use of PROGMEM for your static strings (and calling cmd_query_P()), it still isn't enough free memory for larger sketches. The following is the compile message generated by the beta release of the Arduino IDE (1.5.4).

Sketch uses 22,376 bytes (69%) of program storage space. Maximum is 32,256 bytes.
Global variables use 1,411 bytes (68%) of dynamic memory, leaving 637 bytes for local variables. Maximum is 2,048 bytes.
 
Now, with the new version of the library and SELECT turned on, the connector consumes only 58% of dynamic memory as shown below. While that is better, it isn't quite where we need to be.

Sketch uses 22,152 bytes (68%) of program storage space. Maximum is 32,256 bytes.
Global variables use 1,197 bytes (58%) of dynamic memory, leaving 851 bytes for local variables. Maximum is 2,048 bytes.
 
If we turn off the SELECT feature with the new version, we get a little better.

Sketch uses 20,736 bytes (64%) of program storage space. Maximum is 32,256 bytes.
Global variables use 1,064 bytes (51%) of dynamic memory, leaving 984 bytes for local variables. Maximum is 2,048 bytes.
 
Ah, now we're cooking. The dynamic memory usage is down to 51%. Much better.

Choosing the Right Board

One of the things users new to the connector have struggled with is choosing the right Arduino board for their project. The connector is a non-trivial library that consumes (relatively) a lot of memory. If you want to write a really big sketch using lots of variables, strings, etc., you will need to use a board with more memory.

This is especially true when you combine the connector with other libraries like those made for some sensors, shields, and more. The combined memory for the connector and the other libraries can consume a lot of dynamic memory leaving you very little to use for your own variables.

While most solve the problem by switching to a Mega board, that has its own issues because some of the pins differ from the smaller (memory-wise, not size-wise) boards. A few Google searches will quickly find solutions to these problems (hint: software serial).

So which board should you choose? I've done some research for you and have compiled a simple MySQL sketch using the new version of the connector on a variety of boards. The following lists the memory usage reports from the Arduino Beta IDE. In this case, the compilations are with SELECT turned off (which is the new default).

• Duemilanove, Uno : 1,064 bytes (51%) of dynamic memory, leaving 984 bytes for local variables
• Leonardo : 1,028 bytes (40%) of dynamic memory, leaving 1,532 bytes for local variables
• Mega 2560 : 1,550 bytes (18%) of dynamic memory, leaving 6,642 bytes for local variables
• Yun : 1,028 bytes (40%) of dynamic memory, leaving 1,532 bytes for local variables

As you can see, using the older Uno-style boards are going to limit your ability to write complex sketches with many variables, logic, etc. However, a Leonardo or even a Yun board may be a better choice especially if you run out of memory on the Uno-style board. Of course, the Mega is the best choice if your sketch is going to be very complex or if you need to store a lot of values in memory or use other, large libraries.

Conclusion

I hope this new version solves many of your problems with memory. I've tried to shrink it down as much as I can without radical changes. I believe the majority of the connection failures and freezes will be solved with this new version. Please feel free to provide feedback on this blog or (better) on the Oracle Connector/Arduino forum.

http://forums.mysql.com/list.php?175

Announcing MySQL Connector/Arduino 1.0.1 Beta

I've completed a new release of the Connector/Arduino. The new version supports a few refinements and a new feature.

• New! disconnect() method - enables disconnect from server. Note: you must call mysql_connect() to reconnect.
• Better error handling for dropped packets. No more random reboots when bad packet appears.
• Library can recover from short-term loss of connectivity. Along with bad packets is a check to make sure what is received is valid making the connector ignore garbage packets associated with a dropped connection.
• Detection of Out of Memory condition. Should there not be enough memory to allocate the buffer for the Connector, you will see an OOM error (enable the serial monitor to see the errors). This reduces random reboots when memory gets too low.

I made this release because a number of people were running into problems with noisy, tenuous, or just plain bad network connections. Also, some do not want to hold the connection open on the database server. This release addresses all of these issues.

Did you say disconnect?

Yes, that's right, you can now disconnect from the server should you want to write sketches that connect to the MySQL server for only a brief period then sleep, calculate the distance to Alpha-centuri, make coffee, etc. It is also helpful for those sketches that will update the database only once every few minutes, hours, or days permitting you to connect, run a query, then disconnect on the interval.

But wait...what about when the Ethernet shield goes wonky?

I have also devised a way to overcome the problem of the Ethernet shield controller going away. That is, if your Arduino looses connectivity for more than a few seconds (about 15-30), the Ethernet shield could wig out and fail to respond. The rest of your sketch will continue to run but calls to the Ethernet library will be ignored (how rude).

So...what to do? In short, we need to reboot the Ethernet shield. You could make a hardware-based connection to the reset button but some have reported problems with this solution. And it is a hard reset for the Arduino too - they are inseparable.

Rather than use hardware, I've devised a way to force the Arduino to reload its software. This won't fix any hardware issues like the reset button will but it will restore the Ethernet shield to proper operation.

Ok, I'm sold. How do I do it?

First, you need a variable and a define to set a threshold.

int num_fails;
#define MAX_FAILED_CONNECTS 5

Next, you need a method that will force the Arduino to reload. In this case, I use the tricky jump-to-zero address code which is sort of like a bootstrap (but not quite). Anyway, it works!

void soft_reset() {
  asm volatile("jmp 0");
}

To use this in your sketch, modify the loop() method (or where ever you put your MySQL connector code) as follows:

Note: this assumes your initial mysql_connect() call is in setup() like I originally intended. Modify the following accordingly if that is not the case.

void loop() {
  delay(1000);
  if (my_conn.is_connected()) {
    my_conn.cmd_query(QUERY_SQL); // <-- br="" goes="" here="" query="" your="">    delay(1000);
    num_fails = 0;
  } else {
    my_conn.disconnect();
    Serial.println("Connecting...");
    if (my_conn.mysql_connect(server_addr, 3306, user, password)) {
      delay(500);
      Serial.println("Success!");
      num_fails = 0;
    } else {
      num_fails++;
      Serial.println("Connect failed!");
      if (num_fails == MAX_FAILED_CONNECTS) {
        Serial.println("Ok, that's it. I'm outta here. Rebooting...");
        delay(2000);
        soft_reset();
      }
    }
  }
}

Notice the counter num_fails is incremented any time the connection to the MySQL server fails and is reset when it succeeds. If num_fails reaches the value of MAX_FAILED_CONNECTS, the sketch will call the soft_reset() method and the Arduino will be reloaded (not the same as restarting or resetting - keep that in mind if you use components that require a true reset to initialize).

So now if your sketch runs happily for a time but looses its connection to the database server for a long period, it will reboot itself and therefore reestablish a connection - assuming the network or server is back up.

Note: this code is in the hello_mysql_reboot sketch in the examples folder.

Enjoy!