IBM Netezza – New Version of NPS and Changes to Netezza Query History Database Load process

April 27th, 2018

Itai Weiss
Information Architect

 

Alexander Goron
Netezza Architect

 

Using the latest NPS version

We’ve been busy lately migrating our Netezza appliance customers to the new Mako appliances they bought. As we upgraded the old appliances to a more recent version of the NPS, we found that IBM has heavily modified the Query History data warehouse load process.

We first noticed it during the Mako Netezza database server setup. We setup the Netezza databases load process to run every 15 minutes, and we noticed that upon the 2nd run they started to fail. We took a deep dive into the Netezza scripts and found that IBM modified them; they are not running in batch anymore, but as a Daemon.

A daemon, as most of you know, is a process that runs in the background. It wakes up based on a timer to check if there is work to do – execute it, commit it and go back to sleep mode.

We quickly modified our setup and adjusted our sql queries. Now, we need to start this only once, and every now and then, we verify that the daemon is still running. We changed our H/A configuration a little as well to support this new approach.

We were surprised to find out that this did not fix the problem. The daemon created bad/reject files, with history records that failed to load. We kept digging and found that nzload reported a successful load, but no records were loaded. We found that IBM changed the way that NPS digests the data loaded and rejected these records. We found a workaround, and quickly got the process to load the data with success.

We contacted IBM and opened a PMR. We later got a response back from them that confirms this bug and all our findings. IBM is aware that this bug affects not only Query History, but any other table with a Float4 or Real field data type. They also mentioned that they are working on a fix. We will test and validate it as soon as it becomes available.

During a service call

A few days later, we got a call from one of our customers. To our surprise, he informed us that he’s missing the data in the Query History database since the beginning of the month. We asked him if he upgraded his NPS version, as we suspected this problem. He replied that he did not, and a quick check confirmed that the NPS version was an old version. As we started a discussion with him, we found out that he had a disk replacement done by IBM. Initially, they wanted to upgrade his NPS version to a more current one, but they ended up using the latest support scripts version only. This new version contains the same bug we saw before.

We helped him clean up the problem and load the old data back. We gave him the SQL below and advised him to run it after every upgrade or service call.

select max(qh_tsubmit) from nz_query_history;

This bug was introduced in NPS version 7.2.1.5-p2 .

IBM defect number is 120949.

Sailfish – IBM Integrated Analtyics System (IIAS) Next Generation Netezza

As Netezza has continued to evolve over the years, IBM has added functionality and speed to the appliance. Now, with the significant shift toward data repositories that can handle data science, adapting to both on premise and cloud needs, while still delivering the same robust processing, IBM has released the next generation Netezza, Sailfish. Sailfish is a significant shift away from the traditional Netezza architecture, as it no longer relies on the FPGA accelerators for its speed. It is designed in a containerized fashion, allowing for transportability of work loads, whether on premise or in the cloud. The bundled Data Science Experience (DSX) also brings a breadth of functionality, expanding the use cases and value proposition of the environment. The ability to expand, within a single chasis, also brings additional value add to customers who want to make a move to the new platform, but may need additional elasticity that the traditional Netezza just did not have.

This week, IBM introduced a new plan for Netezza customers. IBM is now offering free migration from a legacy Netezza to the Sailfish. The migration/upgrade to Sailfish is not significant for most customers. But, for those who use stored procedures and other NZ-specific functions to a great degree, it may require a more significant effort for the conversion. The offer by IBM removes this hurdle for customers with complex environments, and it should be capitalized upon. If you are interested in discussing Sailfish and/or your organization’s data needs in more detail, please reach out to the Mainline Information & Analtyics team, your Mainline Account Executive directly, or click here to contact us with any questions.

Do not delay, as we don’t believe IBM will continue this free migration offer past Q2 of 2018.

Submit a Comment

Your email address will not be published. Required fields are marked *