Watch it: Agile COBOL 2009

Thanks to our friends at Micro Focus for THE COBOLATOR video, which they shot to celebrate 50 years of COBOL.

We at MAKE spend all of our time helping customers who are sick of their legacy systems and the lack of flexibility they bring. These mainframe applications cost a lot of companies massive amounts of money. And the problem is not COBOL per se, although 50 years of building software systems has actually resulted in some more suitable programming languages, the problem is legacy systems misaligned with the business they are supposed to be supporting. Running that system on Unix, or Windows, or the Cloud, will not help you with that.

And that’s why you need… Agile COBOL!

Enjoy:

Wang Freestyle

Great introduction video to the Wang Freestyle:

Favorite quotes:

• “First, I turn it into a piece of paper”
• “Get it together guys, get it together”
• “Notice that you can kinda read them”
• “This shows me that this file is stapled together, so I grab my ‘un-stapler’”

The Wang Freestyle cost 1.2 M USD in 1990 and included a phone system which you could not use to make calls, only annotate photographs of files.

The COBOLATOR. Oh my…

Thanks to our friends at MicroFocus for this:

So what skills would THE COBOLATOR have:

• Hunt down executives and force them write copy module code?
• Bill, by force, monthly fees for a platform he no longer intend to support or develop?
• Talk you into moving your legacy mess on UNIX or Windows and make you believe it is modernization?
• Duck every time you ask for a change to the system?
• Convince you lower case is an invention by the devil, and UNICODE its evil twin?
• Battle THE GOVERNATOR and stop him from paying his employees?

Why Is Manual Data Migration Still An Option?

“The six of them fought a battle of attrition against an insurmountable foe; a flow of paper that never seemed to end. Each morning they’d retrieve a big stack of printed customer records, which would represent their work for the day. There was another staff member dedicated to printing, and yet another whose sole task was to staple the records together.”

You can see the rest of this story here: http://thedailywtf.com/Articles/The-Manual-Migration.aspx.

It is surprising how often we encounter clients and competitors who are just not worried about data migration when re-architecting their legacy system. I think the underlying assumption is very similar to the company Joe in this story works at: in the worst case scenario, we’ll just hire a couple of interns and have them transcode the data. Which might be preceded by some SQL script development or some DBA wizardry.

These are organizations that are spending enormous amounts of resources to creating a fully functional, auditable and correct, and at the same want to trust some school age kids to enter their critical data. Or a bunch of Perl scripts which might be right, might be wrong, who knows?

<plug-alert>

MAKE Technologies has a most excellent rule based data migration tool, that audits and logs every data activity that was performed. It runs integrated with the continuous integration build of your application on multi-Gigabyte data-sets. That’s how quick it is. It is called DMW, for Data Modernization Workbench, and more information can be found  here: http://www.maketechnologies.com/products/tlmdm/.

</plug-alert>

- Mik Lernout

Web 2.0 takes revenge

We’ve seen a lot of legacy vendors come out with strangely out-of-place extensions to keep their solutions “current” or “in touch with the market”. We’ve had AJAX on Natural/ADABAS, Object Oriented COBOL and Unisys mainframes on the web, which I added just for its awesome name: Cool ICE, or Internet Commerce Enabler. 

Now Web 2.0 takes revenge: a Javascript API to 80-column punch cards: http://www.outstandingelephant.com/jcquard/.

- Mik Lernout

A Legacy System MAKEover?

With the TV programming spectrum being so spattered by reality shows I thought of making a simplistic analogy between the work we do here at MAKE and the “Makeover” series that you can find on different channels about so many different things that need to be modernized, improved or renovated for whatever reasons.

Legacy MAKEover?

I have to admit the one I like to watch is Restaurant Makeover. If you haven’t seen it yet, it kind of goes like this: The owner of a beat up and dull restaurant realizes that it’s time for a change, time to modernize and transform this joint into a five star restaurant. Wow! Sounds brilliant, but the task is not an easy one, especially if the owner in question has run this business for years without really adding much value to his/her customers. Suddenly, the owner needs architects, construction workers, designers, and of course, the restaurant’s chef also needs to learn new tricks, so there is a learning curve to go through and knowledge transfer that has to happen. Otherwise, what’s the point of doing all this? So, what does the owner do? He calls the experts! And the channel’s seasoned experts are a team that includes an interior designer, a crew of construction workers, and a chef. The team, of course, is not always the same but they are really good because they almost always deliver with the client’s complete satisfaction. They also seem to have lots of fun while they’re at it. Oh, and one more thing: The channel matches dollar for dollar the owner’s contribution for the makeover. Once the owner is convinced to move forward, the project starts.

In the early phases of this makeover, the construction crew smashes counters and walls, rips electrical fixtures, wrecks floors, shatters mirrors, pretty much destroys the place entirely. Then they bring in the poor owner just to check out how things are going. Their eyes almost pop out, jaws drop, and sometimes they even cry. Others just smile nervously (I imagine this morbid part is the one that scores the highest ratings). The process continues by re-building, re-painting, re-decorating and putting all the new fancy stuff in the right place.

It is really interesting to see the owners’ reaction once they see the transformation after completion. But that’s not all; besides the new modern look, the place now serves top notch food which is now prepared by the local chef who at this point should have learned a lot from the channel’s mentor. And the most incredible thing is that everything is done in one week.

Ok, so what are the similarities with our methodology for modernizing a Legacy System? Well,

1. A client decides it’s time to change an expensive to maintain aging system and chooses MAKE to modernize it.
2. We don’t match dollar for dollar the client’s contribution unfortunately. Hear that mighty TV networks?
3. The seasoned experts are assembled (Project Manager, Data Modelers, Architects, Business/System Analysts, Developers, etc)
4. We don’t smash, rip, wreck nor shatter the old mainframe (although sometimes we’d love to… juuust kidding) but we ask, listen, watch, parse, collect, analyze, model, document, draw, organize, and basically capture everything about the legacy system and put it in a data repository.
5. We have lots of fun while we’re at it. If you don’t believe this get in touch with me and I’ll bring you to one of our status meetings so you can attest.
6. Our clients have access to see the evolution of the entire process without the drama, which means no tears so far (that I know of).
7. We don’t re-build, re-paint, or re-decorate. However, once we are confident about our knowledge of the old system, we design and develop it putting fancy stuff in the right places. We call this streamlining and enhancing.
8. We transfer our knowledge to the client’s staff so that they can take over when we’re done.
9. We also deliver with client satisfaction (yes, including oohs and aahs) but…
10. We certainly don’t do the work in a week. Although we’re pretty fast for software development standards.

Conclusion: There are some approximate similarities between a Restaurant Makeover and a Legacy System Modernization. However, if there is any network out there who wants to cash in with yet another reality show please contact us, although I’m afraid we won’t help making your audience ratings soar. Oh well.

- Alberto Rodriguez

Zen of the Green Screen: A Collection of Mainframe Haikus

We at MAKE Technologies spend far too much time thinking and talking about mainframes, the crazy languages they run and legacy issues. These are the results of a mainframe-haiku-contest we held last week, in preparation for some awesome SWAG we’re planning for our Oracle OpenWorld booth at Intel’s Inside Innovation Showcase:

Sun rises in West
Birds fall from sky, hordes panic;
Hide from the mainframe.

Such an old system
It runs your business somehow
A constant worry

Flowers in spring,
Mainframe with empty meaning;
old is new again.

A buzzing mainframe:
sorts, moves, sorts, exports
Finds stillness.

JCL calls COBOL
looks up empty records in DB2
A lotus blooms.

Lonely language;
the last developer turns off the
green glow. Silence.

Batch can not sleep
MOVEs age into variable
sleeps again.

Process records at night
More every day, can’t keep up.
Batch window widens.

Without a lamp
the moonlight turns
my mainframe black

Garbage in:
code converts 4GL to OO;
garbage out.

Ancient, forgotten, cobweb
Unraveled, unscrambled, untangled, unknotted
Meet Tiger Mustang Dolphin.

Batches! JCLs!
How will they fix all this mess
When I have retired?

Mainframe costs soaring
Still the business asks for more.
We will get you off.

CIO demands
Service Orientation!
Show me the money.

Opened the door to
Modernization project.
Now the cat is gone.

Running on mainframe
Like beating a tired old horse.
We will get you off.

Still patching old code
Like a leaky old rooftop?
We will get you off.

Snow on silver hair
Old hands stiff under green screens
Y2K Redux

Hal, open the file
Hal, open the file, Hal
open the, please Hal

COBOL days of dust
Young minds never die
Old is new again

Batch Job override
Transaction code is long lost.
Where is that post-it?

require upgrade;
$time = now() || $never;
$modernize || die;

Sound of one hand clapping
The legacy system was
Enter MAKE and get the applause

How to Win Friends and Migrate Data

Migration is not just for the birds

Data migration is like everything else – there is a right way and a wrong way to do it. Actually there is a right way for every scenario, and many, many wrong ways! This article will help you consider the alternatives.

What is data migration?

Just to make sure we’re comparing apples to apples, we had better define our terms. At times like this, I turn to Wikipedia:

Data migration is the process of transferring data between storage types, formats, or computer systems. Data migration is usually performed programmatically to achieve an automated migration, freeing up human resources from tedious tasks. It is required when organizations or individuals change computer systems or upgrade to new systems, or when systems merge (such as when the organizations that use them undergo a merger/takeover).

Can you imagine doing this by hand?? Nowhere in the world are there workers desperate enough to take on that task uncoerced. Hence the many automated tools on the market – computers will do anything.

What to look for

When you’re looking for a solution to your data migration problem, make sure that you consider a few key features that can make the difference between success and disaster.

1. Enables schema changes

At the end of the day, you want your mission-critical data organized in a rational way that makes it easy to get the data out in a form that serves your business. Trouble is, most legacy data stores are anything but rational. They have grown organically over the past 20 years like English Ivy next to a leaky nuclear reactor – not in an organized way, but with a certain craziness. They tend to exhibit “archeological layering”, so you can see where a certain field switched from use A to use B without actually changing names. The bottom line is that if you have a chance to fix your underlying data model, you should do it. It will make your life much easier in the future. Look for a toolset that allows a flexible and powerful assortment of transformations, including table splitting and merging, rationalization of foreign key relationships, extraction of value lists, etc.

2. Declarative

I hesitate to trot out the comp-sci jargon, but you want a data migration tool that stores your transforms as a set of rules, not as a script. Writing a bunch of scripts is kind of the old fashioned way to get the job done. And sometimes those scripts can get pretty huge and ugly. After all, it’s a programming task in itself, and not a pretty one. Do yourself a big favor and use a specialized tool that has some understanding of the fundamentals of the problem and stores your migration rules as rules, not scripting statements.

3. Iterative

Admit it – you will not get this right the first time. Trying to get it right the first time would be a huge waste of time, and probably lead you down the garden path to “analysis paralysis.” You need to get something done, see how it worked, learn from it, learn your lessons, make changes and do it again. Lather, rinse, repeat. You want a toolset that lets you run your data migration, inspect the results, and facilitate the iterative refinement of the script.

4. Auditable

In some environments this is not a nice-to-have, it’s the law. We can’t go losing our patient drug allergy records or our policy line items or (God forbid) our deposit and withdrawal records! And even in less regulated niches, it’s a good idea to be able to show where every record came from, and where every record went.

5. High performance

This is probably obvious, but when you’re cutting over from the old environment to the new, you will not have weeks and weeks go do it. You’ll be lucky to get 24 hours. And you may have hundreds of millions of records to read, cleanse, transform, re-insert, validate and audit in that all-too-brief period of time. Your tool had better be up to the task!

Integration: the killer feature while modernizing

It’s hard to overstate how useful a good data migration toolset is while you’re modernizing a large legacy system, especially if it’s integrated properly with your modernization toolset.

The first huge benefit is test data. While you’re developing, it makes a world of difference to be able to try out your new screens, batches and reports with actual (not just realistic, but actual) data. So the earlier it’s available in the life cycle, the better. Woe betide those who leave data migration to the end of the project, for they have missed a big opportunity.

Integration is a two-way street as well, because it’s extremely valuable to feed the inevitable screen and interfaces changes back into the modernized schema and the transformation rules, so that you can keep your test data happening with limited fuss.

Yes we built one

I’m sure you will not be shocked to discover that MAKE has just such a tool, designed from the ground-up with data migration in mind, integrated with the rest of our modernization suite. But don’t take my word for it – use your new found knowlege and perspective to evaluate all the data migration tools on the market before making any kind of decision.

- Tom Metzger

How to build a roadmap for your legacy applications

You got to know when to hold ‘em
Know when to fold ‘em
Know when to walk away
Know when to run
- Kenny Rogers

Today I am going to write something simply useful. And I might get in trouble for this, when the execs realize that I’m telling you how to do most of an SMR, a service that we charge for. Anyway here we go – let me set the scene.

At the Crossroads of Modernization

There comes a time in the life of every CIO and IT Manager when they realize that they have a bunch of legacy software applications, and they have a nagging suspicion that they need to do something about it. Often in this moment of truth, after a few rough nights dreaming of exploding mainframes and those horrible “bombshell” meeting when your outsource partner tells you it’s going to be much more expensive than they thought to add a trivial but time-critical enhancement, the CIO will hire an outside consultancy firm to conduct a big study that explains all the various options for modernization, the pros and cons of each, and to create a multi-year modernization roadmap in all its gory complexity.

So I think I can save you some big bucks here, because it’s not rocket science folks! Spend ten minutes running your legacy application portfolio through the following decision tree, and you will be most of the way to a roadmap.

When To Do Nothing

Ah, everyone’s favorite solution! You can do nothing if you still have time to react if something goes wrong. Keep in mind that doing something can be a multi-year process, so the decision to do nothing requires some considerable foresight. The litmus test for this one is simple: are you sleeping well?

When To Kill It

“Retire” is the euphemism here, but your system will not be heading off to the beach to read paperbacks, it will be gone. And good riddance, because if it fits into this category, it was just sucking up your money and giving nothing back. Freeloader.

When To Rehost It

Do this if the mainframe is the real problem, and you just need the same system, in the same language, running on a cheaper box. And even if the system itself is a nightmare, at least you’re not making it any worse. So you can solve your short-term mainframe problem, and deal with the more difficult software issue later.

When To Wrap It

Wrap or “SOA-Enable” your system if the biggest problem is integration. This can extend the life of your system by allowing other things to talk to it. Of course, you’re not getting rid of the system itself, so if the legacy system is a problem, it’s a problem you still have. And it’s also pretty clear that you have increased the complexity of the overall system, and introduced dependencies between the legacy system and the rest of your infrastructure. So it’s not a good way to address issues of complexity, maintenance cost, etc.

When To Port It

You port a system to a new language if the language is the issue. I think of porting like a meat-grinder – you take your old source code, run it through the meat-grinder, and you get the same system in a new language out the other end. Warts and all. You can’t make any material changes to the structure or design of the application, but at least you can get rid of your aging, expensive COBOL or PL/1 programmers and replace them with fresh college recruits.

Except they will all hate you afterwards. The guys you fired will hate you for firing them, and the new Java programmers will hate you for making them work on a Java system that looks like COBOL. Most of them would rather push bamboo under their fingernails. Pay them well, ’cause they won’t be sticking around because of the sexy tech.

When To Rearchitect It

This is what you do when the legacy system is in a sorry state, but still runs your business and has a lot of value buried deep inside. You want the baby, but no bathwater, thank you very much. This may be the most ambitious of the alternatives, but in many cases it’s the only one that gets you what you want out the other end – a nice looking system that doesn’t reflect the pathology of the legacy system it’s replacing.

With a proper rearchitecture, you can get a brand-new, relational data model, a modern tiered architecture, you can decompose the system into SOA-enabled components, you can revise the screens and the use cases to meet new business requirements, you can convert from batch-orientation to a real-time paradigm, you can choose a great new platform and a new architecture that can even include one or more off-the-shelf components, and you can do all that more cheaply and with far less risk if you leverage the legacy system rather than starting from scratch.

As you can see, for many organizations, rearchitecture is the Holy Grail of modernization. And it really hasn’t been possible until very recently, so you can forgive yourself for not knowing.

Conclusion

So now you are equipped to write a roadmap! We’ve written lots of them, and it’s mostly common sense. But let us know if you need any help.

- Tom Metzger

The Learned Helplessness of IT Managers

This morning, while riding in to work, it struck me that managers in large IT organizations exhibit something called “learned helplessness.” From Wikipedia:

Learned helplessness is a psychological condition in which a human being or an animal has learned to act or behave helpless in a particular situation, even when it has the power to change its unpleasant or even harmful circumstance.

Skinner Box: Today's IT Manager's office?

I’m no psyche major, but I know that they did research on this topic in the 50’s by subjecting dogs to electric shock in a lovely device called a Skinner Box. One group of dogs was given no control over the painful shocks – they appeared to start and end at random – and those dogs got pretty screwed up. After a while they didn’t even move when the shocks came, even if they had an easy way to escape them. They just whined.

Pretty horrible stuff, but what does this have to do with modernizing legacy software?

Well, IT managers have had a lot of shocks over the years, in the process of trying to rid themselves of their legacy software applications. Failure after failure after failure.

“Oh, the project is 200% over budget?” ZAAP!

“What do you mean SAP can’t be configured to handle our data model?” ZAAP!

“You must be joking – the vendor went out of business?” ZAAP!

“We can’t find the performance bottleneck in SQL Server, and the system has been down for two days.” ZAAP!

After a while, the poor bastards responsible for modernizing just throw up their hands and say “we’ve already tried that, and it doesn’t work.” They just lie down and whine, deeply suspicious of anyone who says they can help, and hoping they can make it to retirement age before the system breaks down entirely.

And this is one area where dogs actually do better than humans (again, from Wikipedia):

There are several aspects of human helplessness that have no counterpart among other animals. One of the most intriguing aspects is “vicarious learning (or modelling):” that people can learn to be helpless through observing another person encountering uncontrollable events (Bandura, 1986).

Isn’t that lovely? This is where a whole group of people decide that legacy applications can’t be modernized because they saw someone else fail at it! Yes, it’s the all-too-familiar “herd mentality” that we humans tend to exhibit. Probably helped our species survive back in the stone age, when things moved slowly. Ah, to be back in the stone age…

So even when things change, there’s a lot of inertia in people and in groups. What if a new approach or new technology made modernization of legacy application feasible all of a sudden?

Therein lies the opportunity for the bravest souls. The first ones to realize that there is a way will be the IT heroes of their age.

- Tom Metzger