Installer Progress Report

Hello. It's been a few days since my last update. I've made some good progress, and I'm on track for my Thursday release with the goals I've set for myself. The main focus of this release will be to incorporate the dependency checking to some degree so that only missing dependencies are installed. With this eventually will come germane dialog screens that inform the user of which dependencies were located (multiple MySQl/Tomcat/etc), and giving them the choice of which one to use, or to specify another one all together. At the very least this release will be able to detect MySQL and Java and proceed as necessary. I'd also like to implement the dialog logic so that the user can select which MySQL install to use, and actually that is fairly easy. Today I broke a lot of ground in way of achieving that goal, actually.

• Screen Logic - I've got all of the dialog logic working so that I'm in full control of which screens get displayed to the user based on the value of any variable of my choice.
• Dynamic Selection - With the help of BitRock support, I've figured out how to populate a drop-down selection menu with variable data representing valid dependency installs (i.e multiple MySQL installations).
• Dynamic Components - Using similar logic to that of the screen control, I can now control which components of the OpenMRS system are actually installed. We bundle all of the depends with the installer, but depending on the result of the dependency check, only the needed ones are ever unpacked (installed) during the install process.

It may only be "three" things, but those three items are huge milestones in the installer. In fact, those three elements are the cornerstone to the dependency checking interface into BitRock; this is the stuff that makes the installer worth it's weight in gold. I'll hit it hard tomorrow evening and transition into Thursday morning preparing for my evening release. By then, I anticipate a lot of progress in a few other small areas such as:

• Moving any dependencies we install to a standard location
• Cleaning up the install folder after installation, not leaving behind any unnecessary files
• Clean up the environment variable exporting so that it is determined by the information that the dependency checker determines or that the user selects.

I'll keep everyone updated, take care.

OpenMRS Visit #2

Well, today I made my second visit to OpenMRS headquarters in Indianapolis. As usual, it was great to see the guys, but also great to be in a professional software development environment. The OpenMRS guys are truly bright individuals, and it's more apparent in person listening to them problem solve and go back and forth with each other trading solutions. While I did get a lot of work done today, there are still a few hangups that I'm trying to iron out. When I do so, I'll have a much improved beta-installer for you all to try out.

Just a little insight into how I work... The beta's I try to get ready for the weekly releases are 1 of maybe up to 3 different installers I'm working on. When I say "different" I mean, they all contain subsets of more advanced functionality that will eventually be part of the final product. I'm currently working on 2 major installers: First, an installer that is ignorant of the environment, but does silent installations of all of the dependencies and configures them silently as well given the data input during the BitRock installer. Second, I'm furthering the development on the dependency checking code and working that into a stripped down installer for testing purposes. Once the dependency checking is "complete" I'll then merge that into the "ignorant" installer, and everything will just mesh due to some logic rules within BitRock that determine what gets executed when, and why. So, while the next update (coming before Sunday) will only be a marginal improvement upon the last (actually, it's a huge improvement), it is only a subset of the work that I've actually completed.

The "delay" with the dependency checking came in that after the last beta was finished, I started programming the dependency checker. Then, yesterday I found out that the mechanism I'd hoped to use to read my dependency checking program's output wasn't going to work, and I had to find a new method. This was also in parallel with having to get some sort of demo ready for my visit today. So, it's been hectic going back and forth between the two installers while always trying to make progress in both areas, but it's really starting to come together. The source code for the dependency checking is essentially complete, and any little things that come up are easily added/modified since it's C++ and I have full control of what I do. As for getting that data into BitRock, I had started experimenting with that yesterday, but ran into some confusing issues. However, after my work today, I have a much better understanding of how to address those issues.

The current updated installer that I'll have out to everyone within the next day or so includes the following upgrades from the previous installer:

• Multi-Input Dialog Screens - You can now enter all relevant data on one screen instead of 4 or 5
• Silent installations of all dependency applications - I use command line switches for the JRE installer, and I've bundled zipped versions of MySQL and Tomcat with the installer. They are unpacked into their proper places, and then I use BitRock's file editing abilities to configure their settings files with the proper data (username/password/port/etc...)
• Setting of needed environment variables - Certain env variables need to be set, such as the path to the JRE, CATALINA_HOME, etc... as well as adding MySQL to the path after it is installed.
• Demo Data - You now have the option to forgo installation of the demo patient data. This is helpful for people that already have OpenMRS running, and just need to install it on another system.
• Port Validation - The installer now verifies that it can bind an address to the port numbers you specify for MySQL and Tomcat. If not, it asks you to enter a new port.
• Improved Error Detection - The installer now correctly enforces illegal empty data (i.e all of the input data during the installation process)

I think that's about it as far as the changelog is concerned. The current hangups both deal with adding MySQL and Tomcat as Windows Services. I got nothing but grief today when I tried to do this, such as error messages that didn't tell me why it failed... This was the main reason I didn't have a demo to show the guys today, and that really bothered me. I wanted to show them what I had done, and I suppose they can still see everything except for it successfully starting the system, but still... very frustrating, for everyone I'm sure. I think the issue may have been linked with the fact that the machine I was using already had MySQL and Tomcat installed as services, but I didn't want to wreck my development machine with a scavenger hunt that may have turned up nothing.

That's all for now, I'll get the update out as soon as possible. I've worked myself thin the past 3 days, even to the point I had to stay home from my other job due to illness, so I've got to step away from it for tonight and put in a LONG weekend of work and get this thing finished as soon as possible. I'll keep everyone updated, thanks.

Installer Progress Report

Alright, today was a relatively productive day.

• First, the dependency code now detects MySQL in the PATH as well as those not in the PATH but in standard install locations. The issue now is avoiding duplicates, and this won't be easy to handle. You see, on UNIX, the 'which' command tells you the full path that is resolved when you execute a certain command. Windows (by my searching) has no such command. This presents a situation in that we don't know which installation the binary located in the PATH points to. It could point to the only installation they have on their system, or it could point to 1 of any number of installations; we just simply don't know. The only real way to do this is if we find a MySQL located in the PATH AND we found an installation in a default location, we then have to check the PATH variable for the presence of any/all of the default install paths that we found. Just the thought of that program logic scares the crap out of me. Again, 90% of my thought/worry/planning/nightmares account for maybe 10% AT MOST of the end-user use-cases.

• Moving on... The installer now supports the choice of installing the demo patient data, or not. The user is presented with a dialog, and based on the answer, the sql script responsible for loading the demo data is either executed or not.

• I also cleaned up the organization of files, both for my own sanity, and for the cleanliness of the install. The next step in this area is adding in the code to clean up after the installation, removing files that aren't needed. I'm not sure there will be a large amount of those, aside from any dependencies that weren't needed (plus the Java installer exe), and maybe the sql scripts, but I can foresee it being helpful if the users had those on hand. I'll just have to poll the "suits" about what they think. A comforting discovery I made while looking into this tonight is that Windows does support a file moving itself to a new location (i.e a BATCH script that moves itself to another location as its last instruction). It also supports self-deletion as well. Given those two, this is a trivial feature to implement once I know what needs to go, or what needs to go where.

Well, I think that's about it for now. I also had a talk with Ben about the MySQL credentials that need to be addressed in the installer, so I have a better grip on that now. The key next few steps in the installer process are:

1. Detect Tomcat - More difficult than the other 2 dependencies.
2. MySQL Creds - Make sure all needed info is asked for, and presented in a clear fashion.
3. Runtime Properties - This goes hand in hand with #2, and somewhat with the dependency checking.

Alright, I'll keep you guys updated. Take care.

Installer Progress Report

Hello all. I took the weekend off for the most part due to a friend visiting from out of town, but tonight I really dove into the dependency checking code some more.

The code is written in C++ in order to break our dependency on the JRE at install time. This makes things very challenging from a programming perspective because C++ has no built-in way of getting feedback from system() commands. That was the nice thing about Java; all of that stuff was built-in. The only way (that I can see) to do it in C++ is to append " > somefile.txt" to the end of any system() command you want to run, and then read that output file. It's a pain, yes, but it does work, and actually pretty well for what I need it to do.

Anyways, currently the code can detect the JRE version and multiple standard-install MySQL versions. "standard-install" is defined as "Installations located in the standard MySQL install folder: C:\Program Files\MySQL" The way it does this is it obtains a list of all files in the standard install directory, and knowing the directory structure, tries to find a MySQL binary in the bin directory. It then logs each valid install to a file that BitRock will read during installation. If no valid installs are found, it logs a single failure. The same goes for the JRE, except it only checks for the standard JRE which is installed to the PATH by default.

My main goal for tomorrow is to support MySQL that is installed in the PATH. I started to do this tonight, but it became really complicated to search for MySQL in the PATH AND multiple default installs. The main reason is that we need to keep track of the path to each valid install so that BitRock has that information to provide to the user so they can choose which installation of MySQL they'd like to use (if they have more than one). I think I've got a better idea of how I can implement this now that the code is more or less complete, so we'll see how that goes. My other goal is to interface this code with BitRock. This will consist of using BitRocks readFile mechanism and parsing the log file that my dependency code generates. Depending on how easily I get that accomplished, I could start taking action on that information and then displaying the proper screens to the user. Although, I think that will come on Wednesday at the earliest.

I'll keep everyone posted, take care.

Installer Status Update

Hi everyone. Well, the past few days have been very productive and very exciting. A lot of tricky hangups have begun to get resolved. First, I'd like to comment on the installer that is already out there.

On my system, the dependency installers wait for the previous one to finish prior to launching. When I tried to create an updated build of the installer yesterday, this was not the case. In fact, the behavior was very sporadic; sometimes there was a bit of a delay, other times they were all spawned at once. If you run into this issue with the installer, you will most likely have to go through all of the needed installers when they come up, but then run the installer a second time, but cancel the installers then. Otherwise, if you didn't have MySQL / Tomcat, the installation couldn't possibly be successful, etc...

I have opted to not fix this issue for a few reasons. Mainly, once the dependency checking is in place, the installation of these applications will happen in a completely different fashion. I've obtained "binary" zipped versions of MySQL and Tomcat, and those dependencies will happen in that fashion. Java will be the only dependency that uses a setup exe, but that is easily handled silently with command-line switches.

So, my focus now will be getting the dependency checking to a good beta maturity so that the rest of the installer can come together. One nice thing I have found out how to do is multi-input dialogs. This means you'll have one dialog to enter all of your MySQL information, and then another dialog for all of your Tomcat information, and etc... for any other data we may need to obtain in the future. This is a big improvement upon the current version in that there is only one input per screen.

My goal is to have some sort of updated installer every week, ideally on Thursday or Friday evening, but I may change this to Sunday to allow me to have some extra cram time during the weekend to complete some work. This week the "update" came on Tuesday, and you would have had another one last night, but due to those issues I touched on earlier, I didn't want to send out a horribly broken installer. I'd really like to know why it decided to behave completely differently this time, but oh well.

So, plan on an updated installer by next Sunday at the latest, next Thursday at the earliest. Until then, take care.

Installer Beta

Greetings! I put in a 10 hour work day today, and I've got a beta-installer working. It does essentially what the current izpack installer does (although the version a lot of you/us were trying to use was broke, you may not have seen it). As I said before I've been focused on handling dependencies or I would have had this beta out a lot sooner. A couple of notes on the beta:

1. It's a beta. I don't recommend running this on a system with any sort of working OpenMRS installation. The reason is that the installer will drop the 'openmrs' database if it exists. I have done next to nothing with the actual OpenMRS system, so I'm not sure what you all may be doing with it, but you've been warned, 'openmrs' gets dropped and re-created with the demo data contained in the sql scripts on the website. Also, if you have Tomcat running, you MUST stop it prior to running the installer. You can "Cancel" the Tomcat install later, but you must make sure Tomcat isn't running.

2. During the install you will be asked for various pieces of information such as: MySQL Username, MySQL Password, MySQL Port, MySQL database name to use, Tomcat Username, and finally the Tomcat Password. Currently, I don't have silent installs of the dependencies implemented; there are a lot of things to address in that regard and it may not turn out to be possible with the way we are currently doing things. Because of this, you will have to enter this data twice. Once in the BitRock install portion, and then again when BitRock launches the dependency installers. If this data doesn't match, it won't work obviously. If you happen to already have these dependencies installed, be sure to provide BitRock with the usernames/passwords/ports that you are currently using and of course "Cancel" out of the dependency installations when they get launched.

3. More so, if you already have any of the dependency applications installed, feel free to hit "Cancel" when that particular installer gets launched at the end. HOWEVER, if you "Cancel" the MySQL installer, you need to make sure that mysql.exe is in your path.

4. Following with that, if you DON'T hit cancel on the installers, the only one that you need to do anything special for is the MySQL installer. The instructions for that installer are as follows:

• Select "Typical Install"
• Skip Sign-Up
• "Configure Server Now"
• "Standard Configuration"
• "Install As Windows Service" AND "Launch Automatically" AND "Include In Path"
• "Modify" but do not allow remote access

Note: If you choose to uninstall MySQL prior to running this installer, you MUST delete the C:\Program Files\ entry for MySQL. Otherwise, it won't accept the username/password you provide during the MySQL installation. Basically, if you want to let the installer load MySQL, uninstall your copy and delete it's program folder. Then, use BitRock to set a username/password and then give that same data to the MySQL installer that gets launched at the end of the installation process.

5. There will be a slight bit of lag/delay as the installer executes all of the sql scripts and deploys the .war file. This is normal. What you will see is a long pause, then a browser will launch and connect to the OpenMRS installation. Then, a script output box will appear. You can just hit "OK", and then the installation will ask you to view the README, and you'll be done.

6. If you get an "out of sync" error when you try to view the installation, you'll need to restart Tomcat. This probably means you didn't read step #1... but I ran into this when I manually dropped the 'openmrs' database and then ran the installer without ever restarting Tomcat.

The whole process is actually really simple, although it may "sound" complicated. The goal of this beta was to install all OpenMRS and all of its dependencies on a completely bare system. However, in my own testing, I commented out the lines that launched the dependency installers since I already had them, so I know that this method does work as well. However, you just need to be aware of the issues I mentioned above when choosing to forgo the dependency installations in light of using your own.

Your feedback is welcomed, but please keep in mind this is a beta, and a lot of the issues will be fixed in the upcoming days/week. Dependency awareness is the next big step for the installer and that alone will solve 80% of the issues. Thanks!

Download

http://resources.openmrs.org/OpenMRS-Installer-Beta.exe

Installer Update

Hello. After my meeting with the OpenMRS guys on Thursday, I was given some milestones to hit. You see, when I first started working on the project, my first plan of action was to improve upon what my mentor had already created. The major downfall of his work thus far was that it wasn't cognizant of the environment in terms of dependency applications. Actually, the installer failed if you had any of the dependencies installed. So, my main focus was 1. How to detect dependencies on Windows/Linux/OSX and 2. How to go about loading them. During this time, I was exposed to many different install builder options. It was at this point I began to look for something other than izpack to build the installer out of nothing more than curiosity if there may be something better.

Anyways, the point is that the BitRock installer that I demoed for the guys didn't do a whole lot. The interface that the user will see was pretty much set in stone, but behind the scenes not much actually got done in terms of installing anything. The reason for this was that I wanted to leave out all the easy stuff, such as launching dependency installers, running the sql scripts, etc... so that I could quickly test the installer while I was learning how to do the more advanced things such as edit files, interact with outside programs, etc. I knew how to do all of the easy stuff, but I just wanted to keep it out of the code until I had the hard stuff handled. Tonight I've achieved 85% of my imposed milestone for this week.

My BitRock installer now launches all dependency applications once it collects any necessary information from the user, such as the MySQL user name (either one they want to use, or the one they are currently using if they already have MySQl), MySQL password, MySQl port, etc... From here, I've been able to take advantage of BitRock's variable system to pass this information to a post-install script so that I can have access to these pieces of data when executing the dependency installers, the sql script, etc... This is a big step because these were some major issues to address. The main reason those variables are important is because they can aid me in performing silent installations of the dependency applications by passing them to said installers as command line arguments.

Once I get that achieved, the next step is to go back to dependency checking so that we are only installing what we need to install. I'm awaiting word back from BitRock in hopes to gain an understanding of the capabilities of their installer in terms of launching external applications during the install process at specified points and then continuing the installer process once those applications have completed. This will enable me to write my dependency checking code and then call it from within the BitRock installer. My application will pass some information back to BitRock (probably by writing to a file), and then BitRock will be aware of what dependencies need to be loaded.

Well, that's where I stand today. Hope this post was relatively easy to follow, I'm sort of scatterbrained at the moment! Take care, enjoy your weekend.

OpenMRS Community

I'd like to talk a little bit about the OpenMRS community because it is just that; A Community. From the first day I joined the OpenMRS chat room on IRC, I started to make friends with the other hopeful students, as well as the mentors.

The funny thing was, after the SoC dust had cleared and we all found out who was in, and who was out, those friendships didn't change. In fact, one student that wasn't accepted (well, he was, but he was accepted elsewhere as well) is still active in the channel daily! We're all still friends, on IRC, on Facebook, on AIM, and if we lived closer, in person as well.

The great thing about it is, we aren't just friends in terms of programming/working on OpenMRS. We are friends for social reasons. I'd say the conversations that take place between all of us, even the mentors, is 50% OpenMRS and 50% life. The mentors have been a great resource for me personally, not just in terms of my project, but for everything. They are grown adults and if I ever have some obscure life question regarding credit cards, bills, taxes, black magic, etc... I can always ask one of them and get a great answer.

To me, this is the definition of a community. OpenMRS is composed of a lot of different types of people all sharing some very similar, and also very different characteristics and skills. Given this, they are all working together towards a common goal, but at the same time have built a social community that will stand the test of time.

Typical Working Environment

I thought it may be helpful for some people to understand what a typical development session would look like if you were to peek over my shoulder. I'll give you a quick rundown of the myriad of applications/websites/editors/etc that I have open at any time during a development session.

1. Firefox

• Java API - A must!
  
• Gmail Inbox - I like to stay on top of e-mail
  
• BitRock Documentation - To make sure I don't re-invent the wheel...
  
• Various example websites - Sometimes an example is worth a thousand words
  

2. NetBeans

• I usually have a few projects open, some being test classes, others my full blown GUI's
  

3. Visual Studio

• This is only used for my XML editing; it does a nice job. Although I could probably do this with NetBeans?
  

4. BitRock Install Builder

• Sometimes I'll close this if I'm not working on the installer XML file, but it's been open a lot lately.
  

5. Xchat

• #java - Always good to idle and ask questions
• #openmrs - Duh...
  

6. Console

• Although NetBeans does a fine job of executing code, I like to enter the commands myself a lot of times because often I'll have to deal with files in nearby directories and what not, and I just find it a bit more efficient to do it all in one place. I do the same thing on Linux in that I'll code in NetBeans and then compile/run in the console window.

Well, that about does it. It can get very "busy" on my screen, maybe I can use some of this SoC money to grab another monitor and dual-screen it... :)

Mid-Term Update

Well, the week of the mid-term is upon us. Ironically, this is when everything has been coming together for the installer. I now have a clear view of how all the different elements of the installer will work together, and the specific obstacles still to overcome. Solutions have been found for a number of previous obstacles, some just on paper, others have begun to be implemented. Please note that this is focused on the Windows installer, the Linux installer is a separate goal and it will be addressed either once the Windows installer is fairly functional, or sometime soon after SoC has ended. Here is a brief TODO in no particular order:

1. Figure out why the BitRock glob file pattern isn't working. This will enable me to configure the users Runtime Properties file. If you have experience in this, please help! Here is a link showing what I'm trying to do as-per the BitRock documentation:

http://bitrock.com/support_installbuilder_faq.html#A8_1

The part to pay attention to is the "files" tag. The way this works is depending on where that entire chunk of XML code is placed in the project's XML file, the root directory is implied. So, for me, I've got a similar chunk of code to that located in the section that handles files in the install directory; in our case C:\Program Files\openmrs. Now, I simply need to substitute text in 1 file in that directory, and I know exactly what the filename is. Hence, I don't need to worry about all of those wild cards and whatnot. I've tried everything imaginable, and it's not working. The file I'm testing this with is located in:

C:\Program Files\openmrs\temp\testFile.txt

So, given that, if anyone knows the glob file pattern to edit just that file, I welcome your help!

2. Handle some design issues with the preliminary installer (the one that handles dependencies). It's very challenging designing a functional GUI in Netbeans, and then having to modify the code to work within the context you need to use it. For example, the code is command-line at heart, and will only pop-up dialogs when user response is needed. This leads to a hierarchy of GUI classes being spawned on top of the command-line engine, and there are some issues with sharing data between these various classes. I put a lot of work into this last night and will pick up here today since it's the area in which I'm set to make the most progress.

3. Dependent on #2, once the dependencies are verified/installed, I need to add some basic lines to the BitRock XML file to export some environment variables. This will take all of 10 minutes, just a messy loose end that can't be addressed until the depends are satisfied.

4. There is a little more development to be done on actually detecting the dependencies, but this is one of those issues that I've figured out how I can do it, I just need to complete the idea. No more problem solving really involved here.

5. A bit of work needs to be done for the Uninstaller. The reason is due to how OpenMRS functions. It's not a typical program in that the Installer doesn't create all sorts of registry keys, and there aren't a whole lot of files associated with it. Additions will need to be added to the BitRock Uninstaller (probably just some simple Java code) that handles removing of the war file, configuration of Tomcat, etc... Nothing too seriously hard.

That's about all I can think of now, but I may be forgetting some things. Hope everyone's project is going great!

Installer Progress Report

Well folks, the wheels are moving again! I've decided to use BitRock for our installer builder as opposed to izpack/nsis/any combination of the two. Check them out here: http://www.bitrock.com

I contacted BitRock and they agreed to provide OpenMRS with a license for their installer builder. It turns the 30-day trial download into a never-expiring full version. Even better, in the next few months when they have some time, they will build a "Custom Stack" for OpenMRS which will handle the dependencies automatically...FOR FREE!

Until then, I've started work on the BitRock installer and hope to have a beta version running by the end of the weekend, but let's say Tuesday to be safe. The big issue will be the Runtime Properties file. I'm having some issues with BitRock's scripting system (very easy to use though, by the way) to substitute text in files based on a given search pattern. It's all handled by basic XML, but the documentation for this particular portion isn't very clear. Aside from that, the BitRock portion is essentially finished, I just need to doctor up the logo images as well as reformat the license agreement / readme to work within the BitRock text display. Then, I'll still have to fight the dependency checking for MySQL and Tomcat.

We've more or less agreed that the website will instruct the user to load Java Runtime first, then grab our installer. We'll provide a direct link to the JRE EXE at Sun, however. This decision came about because Java code is the easiest way to check for depends on Windows machines, so we really need to have this as a pre-prerequisite.

I'll keep you updated and hope to have some screen shots for you soon, thanks!

Status Update

Hello again. Sorry for the delays. I've been brainstorming with my mentors and Paul about some of the intricacies of the Windows installer. The nice thing about Linux is that dependencies can be checked via built-in command line utilities in a very straightforward fashion. On Windows, however, that is far from the case. The abilities of DOS BATCH scripting range somewhere between 0 and Useless.

While looking at some of the scripting options out there that have been designed for installer applications, I was again not pleased. NSIS seems to be a popular one, but the scripting language is really complex (and un-necessarily so). This led me to just cook up some Java code that executed the version commands of each of the depends and then did some string manipulations in order to extract the version numbers. This is the same way the Linux version handles this issue. Also, in this process, I'm aiming to stay away from looking at the Registry because that would be a downfall of XP to Vista functionality. For example, the Java entries aren't even remotely the same between the two operating systems.

The real problem will come with MySQL and Tomcat because 1. MySQL doesn't add itself to the users path variable, so we can't directly execute the command and 2. Tomcat is really difficult to check for version info; even on Linux it's a bit hackish but effective. The issue will be trying to achieve that same goal on Windows, but by looking at the files that Tomcat uses on Windows as compared to Linux, that may not be possible.

So, that's where I am for now, sort of spinning my wheels at this point looking for the best solution. I like to think out my problems before getting too involved in actually coding something up as to not waste any effort, but nonetheless progress is being made. I'm gearing up for my visit with the OpenMRS guys next week, so that should be a very productive day and I anticipate a lot of working getting done there. I'll keep you guys posted, take care.