11/24/09

Installing Windows XP on a Tablet PC with an SSD Drive

Is it possible to escape the lure of upgrading a laptop with a shiny new 128 GB SSD drive? Of course not! I am a big fan of virtualization and I am running a development stack consisting of tools like Rational Software Architect, InfoSphere Data Architect, WebSphere Application Server and DB2 9.7 Enterprise Edition in a VMWare virtual machine. My hope was to improve performance of my laptop by using an SSD drive.

To reduce the amount of gunk that accumulates over time, I decided to avoid restoring the content of the drive from my backup and instead do a fresh install. I replaced the hard drive on my Toshiba Portege M400 tablet with a 128 GB Patriot Torqx M28 SSD drive and started installing Windows XP from the MSDN distribution. However, the installation was not without problems and some research was necessary: the precise set of steps to get things working is not explicitly spelled out elsewhere. Same or similar steps should be applicable to other laptops and tablets. If you need to do this, I hope you will save yourself some time and frustration by following my advice!


How about Windows 7?

Good question! It turns out that my tablet is officially just a bit too old to run Windows 7. I was hesitant to go to Vista for its known problems and performance degradation in comparison with XP. So I decided to run XP just little bit longer, until tablets get quad core CPUs and 16 GB RAM ;-) When I teach and consult on software architecture and design I draw a lot (so I need a tablet), but I also run heavier stuff like WebSphere Application Server (so I need power) when I consult and teach things like SOA and Web Services.

Installing the SSD: Hardware is easy!

This was the easy part: unscrew the disk compartment, take out the old drive and put in the new one. The SSD drive behaves like a SATA drive it replaces. It fits nicely.

The Missing SATA Driver

Pop in the XP CD, start the setup and there is the first problem: XP does not recognize SATA ! The way around it is to provide a SATA driver on a floppy during the setup. Not USB, not CD or DVD - it must be floppy. Emulating floppy with USB drive did not work. I did not see a floppy drive for many years and my laptop did not come with one. What now?

The solution is to "slipstream" the driver using the nLite – the Windows installation customizer. (Check out their guides and forums for detailed guidance.) You also need to download the SATA driver from your computer manufacturer site. In my case, this was the Toshiba RAID driver. During the preparation of the nLite install, you provide the location of the unzipped driver, which gets integrated ("slipstreamed") into the XP. Next you should create an ISO image and burn it on a CD.

BIOS and the Lonely Drive RAID

As this was the only drive on the machine, the BIOS of my laptop was showing that the drive is in a JBOD setting. If you leave BIOS in this setting, the XP with the slipstreamed driver would still not recognize the drive! You need to go to BIOS and change the drive configuration to RAID – it does not matter that this is the only drive on the machine. Somehow, this setting makes some sense, as the driver is for RAID.

The MSDN Path Complication

If you are using MSDN distribution for installing your XP, you have a CD that contains XP Home, Professional, and Tablet versions on the same CD, but in different directories. The XP installation script has references to the location of files, which are now wrong for the nLite installation. In the directory where nLite copied the XP files, you need to go to the installation configuration file in i386\txtsetup.sif and modify the setup path SetupSourcePath to "\".

A Case of a Missing File

If you try to install now, XP will throw an error message about the missing EULA file and abort installation. The funny thing is that the file is in its proper location. This problem can be fixed by defining the installation type in nLite as "silent". Obvious ;-) You will need to provide the software key in an nLite configuration form.

The SSD Formatting Problem

At this point you may think you are in good shape: XP recognized the drive and you are proceeding to the point where you need to format your hopefully blazingly fast SSD drive. But you are here for a new surprise: XP reports that there is not enough memory to format the drive! To cut the long troubleshooting story short: XP setup cannot format the SSD drive. You need to format your SSD drive and create a partition using a separate working computer. That worked fine and I put the formatted SSD drive back into the laptop. Now I restarted the XP setup and when we reached the troubled point it was important not to create a new partition, but to use the drive as is. The XP finally starts installing and copying files to the SSD. Yay!

But where is the Tablet Stuff?

OK, now we have everything working – but there are no tablet features in sight! In fact, the MSDN tablet components are on the second CD, and we were supposed to be asked to insert it at some point, but that did not happen. Unfortunately, there is no easy way to retroactively add the tablet features. Here is a trick to get the tablet functionality:

  1. Boot XP
  2. Insert the Windows XP MSDN installation CD
  3. Run setup program from the CD
  4. Choose the "update" option
  5. Setup is basically reinstalling XP, but is using the existing SATA disk driver on the machine
  6. At one point it will ask you for the second CD. Insert it.
  7. XP update completes and you have a working XP tablet.

At last: Success and Blazing Speed!

So was this hassle with SSD and XP reinstallation worth it? With all my programs installed, the boot time of my machine went down from over 4 minutes to 25 seconds - that is until CPU got steady on 0%. Starting a 22 GB VMware image now takes 25 seconds - 2.5 times faster. Starting Rational Software Architect in that VMware image improved 4.5 times. Wow! Now we can get faster to doing something more interesting than installing old operating systems!


Have fun and good luck with your upgrades!



11/9/09

Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery

In this IOD 2009 talk I spoke together with Dan Galvin on using pureQuery: a new data access platform with some unique features. pureQuery is a practical technology that strikes the balance between high productivity of developers and providing a full control of the database access by not hiding the SQL. These features make it particularly useful for critical enterprise applications where high performance and large amounts of data dictate special attention to the data access layer and where we have witnessed many object-relational mapping technologies underperform. As a leading DB2 z/OS expert, Dan Galvin shines light on some important performance issues of data access technologies for large, mission critical systems using DB2 on System z. We particularly highlight the impact of static SQL and heterogeneous batch, which are missing in other data access solutions.

Web 2.0 Development with IBM DB2

Presentation slides from this year’s IBM Information on Demand conference are now online! I spoke together with Rafael Coss from IBM on exciting topic of creating Web 2.0 applications with WebSphere sMash, Groovy and Dojo framework. While there are many technologies out there for creating such applications from scratch, we address a real problem where we deal with large amounts of data and an already existing database. We show the combined use of Optim Development Studio and pureQuery for efficient access to DB2 and WebSphere sMash for rapid development.

10/8/09

The Role of pureQuery in Legacy Modernization: From DB2 COBOL or PL/I Applications to Java – One SQL Statement at a Time

The Unavoidable Task of Legacy Modernization
New technologies often lure us to write programs using such cool features like metaprogramming in Groovy, Erlang style parallelism in Scala, Hadoop clusters, or the power of our freshly developed Domain Specific Language. However, every now and then we face a serious task of migration a legacy system from COBOL or PL/I to Java. The languages themselves are old, the developers are difficult to find and there is usually an organizational mandate to move from legacy language to Java. The application that needs to be replaced, albeit elderly, may be a critical one, processing billions of dollars a year.

Challenges in Language Transition
Modernizing legacy systems is sometimes done by external teams that are proficient in Java. However, such teams often do not have the understanding of the application whose replacement they are creating. The unfamiliarity with the problem domain often results in a new application being an inadequate replacement. Some organizations, aware of this problem, engage original developers, after they receive Java training and mentoring. The idea is that they will be able to rewrite the old system in Java, having the advantage of understanding the legacy solution and the problem it addresses.

However, COBOL or PL/I developers trained to program in Java have difficulties in the migration. The syntax of the new language is not the key problem; it is the new way of thinking. There is a danger of developers using the Java syntax and the old way of thinking about development. These challenges are widely known and discussed in many articles in industry and academia.

Challenges in Database Access: Productivity, Performance, Security
There is one more difficulty that is not receiving enough attention: the difference in the way how database access is performed in Java and in legacy systems. Most legacy applications use embedded SQL, with convenient mechanisms for passing data between host language and embedded SQL statements. The migration to Java exposes the developers to JDBC, and its low level APIs which require many lines of code to accomplish a simple task. A common reaction to this is that the developers question how this new technology is better than legacy when they are forced to write more code to accomplish the same task. The JDBC database access is simply less productive then a legacy technology. There was an attempt to remedy this situation by creating embedded SQL for Java known as SQLJ, but this solution never gained much popularity. Various object-relational mapping solutions like Hibernate or JPA improve productivity, but at a cost in performance and less than ideal solution when a database already exists. Notice that an often touted performance benefit of cashing, provided by some object-relational mappers, does not help if the applications do not try to access the same data multiple times.

But it does not end there: when developers run their programs they are in for a further surprise: the new program runs slower than the new one. An examination reveals that the legacy systems use static SQL, while JDBC uses dynamic SQL.

When using static SQL with DB2, the development tool sends the SQL to the database during the development time. The SQL can be optimized at the highest optimization level, taking into considerations the properties of data actually stored in the database. On DB2, this compilation and preparation process is called binding. The applications at runtime execute this compiled SQL resulting in fast execution. JDBC applications use dynamic SQL: the SQL is sent to the database first time during the execution of the program. The level of optimization is much weaker, as the database does not performs a thorough optimization as it may take too much time. Albeit a database can use a cache of used SQL statements, this is still less efficient than running highly optimized, precompiled static SQL.

To dismay of DBAs, JDBC can send any SQL to the database for execution. Contrast this with static SQL, which can use a number of security controls restricting the SQL that will run against the database. A low hanging fruit of static SQL is the immunity against the SQL injection attacks.

Legacy Migration with Optim pureQuery Runtime and Optim Development Studio
Optim pureQuery runtime and its APIs allow us to be productive in the migration to Java and to achieve good performance and security. While there is more to migration from legacy to Java than just the database code, we see that this is often a critical part of an application. In migrating legacy to Java, we have noticed a pattern for effective migration using pureQuery persistence API and Optim Development Studio tooling. Here is how to accomplish it using Optim Development Studio using a seven step process.
  1. Create a Java project in Optim Development Studio. This is where your Java code will reside.
  2. Create a Data Development project. This is where you will collect and work with SQL from the legacy system.
  3. Find the SQL in the legacy application and copy it to a SQL script in a Data Development project. You may want to run the SQL in order to ensure that you understand what it is doing.
  4. Generate pureQuery code using the pureQuery. This will result in creation of annotated data interface that contains SQL code in annotations. Query results are passed through generated Java beans. The implementation for the interface is generated as well.
  5. Repeat step 4 for all SQL statements from the legacy application.
  6. Implement application logic in Java, using the generated data interface and beans for passing the values in and out of SQL. As an additional boost in performance, developers can arrange the execution of their code so that it uses heterogeneous batch – significantly reducing the number of interactions of the applications with the database: various SQL statements are accumulated and then send to the database in one interaction only. Notice that some object-relational mappers can send the SQL to the database at the end of transaction, but this can still incur the same number of database interactions as before.
  7. When ready for deployment, bind your application from Optim Development Studio. During execution, the application can then use static SQL on DB2.
The Advantages of pureQuery over Coding Database Access with JDBC
This approach to modernization of legacy applications has several advantages over coding with JDBC.
  1. Productivity. Optim Development Studio generates complete pureQuery code for database access. This feature is very much appreciated by the legacy systems developers who rightfully find JDBC code to be error prone and difficult to write. Even if we would not use the convenient Development Studio tooling, hand written pureQuery code is significantly shorter than equivalent JDBC code. In addition, when editing SQL from Java code, Optim Development Studio provide for SQL autocompletion for tables and columns using a live database connection.
  2. Performance. Performance is improved in two ways: first through the use of static SQL, and second through the use of heterogeneous batch. In our experience, when accessing a database residing on a different machine from the application, heterogeneous batch has a decisive performance advantage over JDBC.
  3. Security. This is due to the ability to restrict the SQL statements run on the database. The applications do not lose the security controls they had in the legacy application.
  4. Traceability and Troubleshooting. Optim Development Studio tooling provides for traceability between the SQL that is sent to the database and the Java statements that contain the SQL. This traceability mechanism provides for better troubleshooting, as problematic SQL can be easily identified in the application code. In addition, for the SQL in the application, one can use performance optimization tools like Optim Query Tuner to optimize SQL, use explanation of access path with Visual Explain and use performance metrics for queries.
Conclusion
We found the migration of legacy DB2 applications to Java is not as difficult as it used to be, thanks to Optim pureQuery runtime and Optim Development Studio. The pureQuery solution fits the gap that exists between cumbersome and verbose JDBC code and object-relational mapping which often underperform when using with existing databases. The presented seven step migration process is easy to follow, even for developers new to Java. The migrated applications execute SQL securely and efficiently, offering a rich functionality for traceability and sophisticated performance tuning.

The conclusion: If you are planning to migrate a legacy DB2 application to Java, I recommend to give pureQuery and Optim Development Studio a try – you might be pleasantly surprised with its efficiency and productivity.

The techniques described in this post are explained in detail in our course Development of Database Applications with pureQuery and Optim Development Studio.

Happy legacy modernization!

8/21/09

pureQuery Course: The New Optim Development Studio Editon

There is a new pureQuery course: Developing Database Applications with Optim Development Studio and pureQuery. The course uses the latest Optim Development Studio 2.2.

If you are still using the IBM Data Studio, the old course is a better match: Developing Database Applications with IBM Data Studio and pureQuery. The new course reflects the updates in tooling and our experiences delivering the course.

For those of you on mainframe machines who are aware of the differences between DB2 LUW and z/OS versions: don't worry! The labs and exercises can be run with DB2 z/OS on an IBM System z mainframe.


8/14/09

Data Modeling Misconceptions and Problems
(...and what to do about them)

In my work with organizations that are trying to introduce data modeling, several misconceptions appear frequently. Modern data modeling tools and productive practices make the data modeling a much easier task than it was before. However, many of the prejudices and misconceptions remain. There are various ways how to deal with these issues, and I will here address just those that we initially encounter when introducing data modeling.

Most of the advice is tool agnostic. However, some of the advice is tool specific, reflecting the features of the tools I used. The tools I was using were:
These tools recently changed names. A handy reference for old and new names is here.

What follows are the top issues and advice how to apply these tools to resolve them. Why is it good to demonstrate the tools? It is because experiencing the automatization and enhanced productivity turn out to be important issues in achieving acceptance of modeling, as just preaching about the benefits does not help. One needs to see the tools in action in order to believe.

Misconception: Models as “Pretty Pictures”
Symptom:
  • Models are viewed as pictures that do not relate to real systems.
Causes:
  • Tradition of modeling only for documentation.
  • Often done with Visio or other drawing tools.

Solution:
  • Show Data Architect features for generation of physical models and then DDL.
  • Reverse engineer a database.
  • Transform models.
  • Show integration with Optim Development Studio and Optim Data Administrator. Exercise continuity in development from data models, to Java application, refactor the database and analyze impact of the database changes to the SQL in the application.
Misconception: Modeling is Hard
Symptoms:
  • Modeling is avoided, instead DDL is written by hand or generated by object-relational mapping tools.
Causes:
  • Lack of training in modeling + familiarity with existing tools.
  • If the problem domain is hard, models are hard to make.
  • Avoiding modeling creates a (temporary) feeling of progress.

Solution:
  • Demonstrate the benefits by showing the continuity from logical to physical models and DDL.
  • Demonstrated benefits motivate learning and inclusion of modeling in the development process.
  • Visualize the models created by hand or by an object-relational mapping tool and point out the problems related to maintenance and scaleability.

Misconception: “Big Upfront Design”
Symptoms:
  • In some agile projects, modeling is rejected on the grounds that complete data models cannot be created up front, as all the application needs are not known.
Causes:
  • Lack of knowledge on how to do agile data modeling.
  • Lack of understanding what modern tools can do.
Solution:
  • Demonstrate the iterative modeling with Data Architect and refactorings using Optim Data Administrator.
  • In code: show impact analysis with Optim Development Studio.
  • Database can be refactored and data migrated.
  • Educate how to do agile data development.

Problem: Analysis Paralysis
Symptoms:
  • Modelers don’t know how to start, no models are created and project descends into coding chaos.
Causes:
  • Lack of knowledge of pragmatic modeling techniques.
Solution:
  • Start from concrete examples, then generalize into models.
  • Work with a domain expert to find and discuss examples. Concrete examples will be needed anyway later for testing. This is a proven technique from domain (or business) modeling approaches. It is best done in a workshop with a small team.
  • Consider prototyping. In our experience we have found Eclipse Modeling Framework (EMF) to be an excellent prototyping tool. We would create ECore models in Rational Software Architect and then use the EMF generator to create the prototype application. The generated application is then used to validate the understanding of data structure by interacting with the prototype.
Many of our experiences in introducing modeling and making modelers more productive went into our new course: Mastering Data Modeling with IBM InfoSphere Data Architect. Check out the outline here.

I will elaborate on some of the data modeling techniques in future posts.

Happy modeling!

8/12/09

Scala Tech Talk @ LinkedIn

Scala is one of the most exciting language developments in recent years. A month ago I was at the Scala Tech Talk at LinkedIn, featuring the creator of Scala Martin Odersky, together with Nick Kallen of Twitter and David Pollak of Lift framework.

Enjoy the video of the talk and the discussion here.

Scala should be the language to master this year!