Wang Freestyle

Great introduction video to the Wang Freestyle:

Favorite quotes:

• “First, I turn it into a piece of paper”
• “Get it together guys, get it together”
• “Notice that you can kinda read them”
• “This shows me that this file is stapled together, so I grab my ‘un-stapler’”

The Wang Freestyle cost 1.2 M USD in 1990 and included a phone system which you could not use to make calls, only annotate photographs of files.

The COBOLATOR. Oh my…

Thanks to our friends at MicroFocus for this:

So what skills would THE COBOLATOR have:

• Hunt down executives and force them write copy module code?
• Bill, by force, monthly fees for a platform he no longer intend to support or develop?
• Talk you into moving your legacy mess on UNIX or Windows and make you believe it is modernization?
• Duck every time you ask for a change to the system?
• Convince you lower case is an invention by the devil, and UNICODE its evil twin?
• Battle THE GOVERNATOR and stop him from paying his employees?

Why Is Manual Data Migration Still An Option?

“The six of them fought a battle of attrition against an insurmountable foe; a flow of paper that never seemed to end. Each morning they’d retrieve a big stack of printed customer records, which would represent their work for the day. There was another staff member dedicated to printing, and yet another whose sole task was to staple the records together.”

You can see the rest of this story here: http://thedailywtf.com/Articles/The-Manual-Migration.aspx.

It is surprising how often we encounter clients and competitors who are just not worried about data migration when re-architecting their legacy system. I think the underlying assumption is very similar to the company Joe in this story works at: in the worst case scenario, we’ll just hire a couple of interns and have them transcode the data. Which might be preceded by some SQL script development or some DBA wizardry.

These are organizations that are spending enormous amounts of resources to creating a fully functional, auditable and correct, and at the same want to trust some school age kids to enter their critical data. Or a bunch of Perl scripts which might be right, might be wrong, who knows?

<plug-alert>

MAKE Technologies has a most excellent rule based data migration tool, that audits and logs every data activity that was performed. It runs integrated with the continuous integration build of your application on multi-Gigabyte data-sets. That’s how quick it is. It is called DMW, for Data Modernization Workbench, and more information can be found  here: http://www.maketechnologies.com/products/tlmdm/.

</plug-alert>

- Mik Lernout

Zen of the Green Screen: A Collection of Mainframe Haikus

We at MAKE Technologies spend far too much time thinking and talking about mainframes, the crazy languages they run and legacy issues. These are the results of a mainframe-haiku-contest we held last week, in preparation for some awesome SWAG we’re planning for our Oracle OpenWorld booth at Intel’s Inside Innovation Showcase:

Sun rises in West
Birds fall from sky, hordes panic;
Hide from the mainframe.

Such an old system
It runs your business somehow
A constant worry

Flowers in spring,
Mainframe with empty meaning;
old is new again.

A buzzing mainframe:
sorts, moves, sorts, exports
Finds stillness.

JCL calls COBOL
looks up empty records in DB2
A lotus blooms.

Lonely language;
the last developer turns off the
green glow. Silence.

Batch can not sleep
MOVEs age into variable
sleeps again.

Process records at night
More every day, can’t keep up.
Batch window widens.

Without a lamp
the moonlight turns
my mainframe black

Garbage in:
code converts 4GL to OO;
garbage out.

Ancient, forgotten, cobweb
Unraveled, unscrambled, untangled, unknotted
Meet Tiger Mustang Dolphin.

Batches! JCLs!
How will they fix all this mess
When I have retired?

Mainframe costs soaring
Still the business asks for more.
We will get you off.

CIO demands
Service Orientation!
Show me the money.

Opened the door to
Modernization project.
Now the cat is gone.

Running on mainframe
Like beating a tired old horse.
We will get you off.

Still patching old code
Like a leaky old rooftop?
We will get you off.

Snow on silver hair
Old hands stiff under green screens
Y2K Redux

Hal, open the file
Hal, open the file, Hal
open the, please Hal

COBOL days of dust
Young minds never die
Old is new again

Batch Job override
Transaction code is long lost.
Where is that post-it?

require upgrade;
$time = now() || $never;
$modernize || die;

Sound of one hand clapping
The legacy system was
Enter MAKE and get the applause

The Learned Helplessness of IT Managers

This morning, while riding in to work, it struck me that managers in large IT organizations exhibit something called “learned helplessness.” From Wikipedia:

Learned helplessness is a psychological condition in which a human being or an animal has learned to act or behave helpless in a particular situation, even when it has the power to change its unpleasant or even harmful circumstance.

Skinner Box: Today's IT Manager's office?

I’m no psyche major, but I know that they did research on this topic in the 50’s by subjecting dogs to electric shock in a lovely device called a Skinner Box. One group of dogs was given no control over the painful shocks – they appeared to start and end at random – and those dogs got pretty screwed up. After a while they didn’t even move when the shocks came, even if they had an easy way to escape them. They just whined.

Pretty horrible stuff, but what does this have to do with modernizing legacy software?

Well, IT managers have had a lot of shocks over the years, in the process of trying to rid themselves of their legacy software applications. Failure after failure after failure.

“Oh, the project is 200% over budget?” ZAAP!

“What do you mean SAP can’t be configured to handle our data model?” ZAAP!

“You must be joking – the vendor went out of business?” ZAAP!

“We can’t find the performance bottleneck in SQL Server, and the system has been down for two days.” ZAAP!

After a while, the poor bastards responsible for modernizing just throw up their hands and say “we’ve already tried that, and it doesn’t work.” They just lie down and whine, deeply suspicious of anyone who says they can help, and hoping they can make it to retirement age before the system breaks down entirely.

And this is one area where dogs actually do better than humans (again, from Wikipedia):

There are several aspects of human helplessness that have no counterpart among other animals. One of the most intriguing aspects is “vicarious learning (or modelling):” that people can learn to be helpless through observing another person encountering uncontrollable events (Bandura, 1986).

Isn’t that lovely? This is where a whole group of people decide that legacy applications can’t be modernized because they saw someone else fail at it! Yes, it’s the all-too-familiar “herd mentality” that we humans tend to exhibit. Probably helped our species survive back in the stone age, when things moved slowly. Ah, to be back in the stone age…

So even when things change, there’s a lot of inertia in people and in groups. What if a new approach or new technology made modernization of legacy application feasible all of a sudden?

Therein lies the opportunity for the bravest souls. The first ones to realize that there is a way will be the IT heroes of their age.

- Tom Metzger

Mainframe vendor doth protest too much?

In this article, we see the PR wing of a large mainframe vendor hard at work to turn back the clock and revive the mainframe age. The article has a defensive tone to it, as if to suggest that since everyone knows that mainframes are in decline for a half-dozen obvious and good reasons, they need to create some counter-spin to protect their revenues. My favorite part of the article was:

“Classic mainframe myths still exist today. Mainframes have traditionally been associated with high total cost of ownership; lack of advanced applications; inability to support real-time/low-latency processing; poor back-end data-integration support; a shortage of mainframe skills; and steep and inflexible development and maintenance curves,” he said. Some of these issues have been addressed, however.

Back to the Future?

Emphasis above is mine. How do you feel when someone tells you, “don’t worry about that – we’re dealing with it.” Are they trying a Jedi mind trick? Doesn’t it make you wonder which issues have been addressed? And what were the results of this addressing? Do mainframes now have low total cost of ownership, or is there no longer a shortage of mainframe skills?

Well clearly not the latter, since they go on to say:

Human resources, though, could prove a challenge. “There are simply not enough young, bright people wanting to learn mainframe skills over PHP, Java, Flash and other ‘hip’ Web 2.0 technologies,” said Sheina.

Yup, that’s what I keep hearing. But hey at least they do have “tie ups” with a whole bunch of educational institutions. That’s a bit ominous.  Does this mean they’re now doing to educational institutions what they do to clients?

If they’re having revenue increases on the mainframe side of the business, one can only assume they have become more adept at applying the thumbscrews to their existing customers. But this is good for the modernization industry, because the more pain their mainframe customers are feeling, the more they are going to want to get the hell off of it, and the biggest obstacle to that maneuver has always been those pesky COBOL and other legacy apps. The mythology around moving those applications is that it can’t be done.

Except now it can be done. The cost of moving those applications to Java or .NET in a useful way has plummeted in the last few years. And that’s not good for mainframe vendors who rely on “vendor lock in” to keep the racket money flowing.

- Tom Metzger

Developer’s Eye on Modernization: Conceptual Model, Please

It’s interesting and very helpful to read the piles of articles on legacy modernization. There is definitely no lack of opinion from architectural, business, methodological and management people. As one of the developers working in the trenches on legacy modernization projects, I’d like to throw in my two cents from the point of view of a POJD, hmm, Plain Old Java Developer.

My first point is this: let the developers in on the conceptual business model and leverage their reasoning power to extract detailed business rules! Why is that? Isn’t the developers’ responsibility to be focused only on the technical part of the application? Doesn’t this area belong to the BAs, SAs, PMs to master the conceptual model? Isn’t it a waste of money and time to teach the techie nerds conceptual business model? No, those premises, which worked well in traditional from-scratch projects are not valid for the “new-generation” of modernization projects.

Behind any effort there is a purpose, or we usually call it requirements in software development. What the purpose of a specific modernization project? Of course, to reproduce the values from an “old” system to a modernized system, in the aspects of programming language, platform, architecture, maintenance; and the list goes longer if timeframe and finances allow. Don’t let the nouns fool you, the essential value to the customers is based on one simple truth, and that is to accurately reproduce the business flows, rules and logic in the new system.

Reasons:

1. Unlike Greenfield projects, the clients don’t have to provide detailed user requirements before the developers begin to implement them. They are hoping, and to some extent they are right, the legacy system itself is the most complete and accurate source of user requirements.
2. Fortunately, the conceptual model is NOT difficult to re-construct if it was nicely documented already.
3. The physical model, however, usually is not fully documented, if at all. They are scattered throughout the legacy code. It’s a huge job, or even impossible, to let the BAs try to extract every detailed rule by communicating and asking clients questions. It’s far from an efficient approach. Some weird business rules are deeply buried in the code, so the client and legacy system operators can’t recall or understand them.
4. From experience a lot of use cases and business rules written by BAs are not accurate, not complete, or simply not understood by developers. Instead, developers can extract and implement those rules by reading legacy code or watch the video walk-through of how the legacy system behaves.
5. The more efficient way is to teach the developers to understand the conceptual business model, then using their reasoning, have them read and recover the detailed business rules buried in the legacy code.
6. This approach implies: developers should be get involved in Phase 2 (Transformation) to understand the business model; Phase 2 can be shortened such that the BAs don’t have to try to extract and document the detailed use cases and business rules, rather they should focus on the bigger picture of essential business model and flow. In phase 3 (Implementation) developers need to read the legacy code to grasp and re-implement the detailed business rules and constraints.

I know it’s difficult for developers to master these two parts simultaneously: business part and technical part. Or even worse, three parts: new technology platforms like J2EE and .NET architecture, dinosaur technologies like Natural and RPG language and hierarchical database, and the out-space business model. I think it’s this extreme on the developers’ brain that contributes to the failures of lot of modernization projects. But the challenge is also the opportunity. By recognizing this problem, researching and training process, talented developers can eventually gain the capabilities to craft a high-quality target system. In the end, it’s all about people.

- Lin Wei, Sr. Java Developer

SAP: How I Learned to Stop Worrying and Love the Bomb

SAP just announced they will increase the maintenance charges on some of their products by about 22%. And when the vendor who runs your business requires you to pay money, you comply.

I saw the rise of ERP, and SAP, first hand during the last internet bubble, working in or close to Brussels, a major capital of the European Union and of SAP adoption. I saw companies burn millions of euros on moving to the platform, wreaking havoc on established, custom and sometimes highly optimized business processes. I saw a company I worked at, called Home Shopping Europe, crash under the pressures caused by an eternal, over-due & over-budget SAP implementation, diverting enormous amounts of cash and effort into what is virtually a commodity. (I should note here that just like any feature, there were more factors involved in HSE going under, but the SAP implementation most definitely contributed to its decline.)

What angered me at that time was rooted in my developers ethics:

Why use those expensive consultants, using expensive tools to build something that didn’t even really ‘work’ in the traditional software engineering sense, especially when a bunch of marginally competent developers, with some marginally competent business analysts can create a solution at a fraction of the time and cost? Also: aren’t our business processes connected to our competitive advantage? If those become commoditized we will never be able to compete on process.

And then, after working in the enterprise content management space for a couple of years, I stopped worrying, and accepted reality: massive COTS ERPs will be running every corporation in the foreseeable future. And what is so bad about that? Custom development is hard, most processes are commodities and if everyone does it, it cannot be that wrong. I started to learn to love the bomb.

When I read about SAP jacking up the maintenance price this morning, the bomb became scary again:

With an economy slowing down, companies start to reach for their built-in survival instincts. ERP vendors, like SAP, will be forced to offset the decrease in new implementations by increasing the per-corporation licensing and maintenance fees. And companies will have to pay up, or their business will no longer run. Some people will call that corporate extortion, and those people would be correct.

An ERP vendor and its client have a parasitic relationship, which is OK when the economy is fun and games, but can turn ugly quickly, when the going gets rough. I cannot think of any other corporate relationships that are so entangled with each other (except for maybe those Facebook application development shops ). Another, ironic, example: Supply Chain Management is all about maintaining control over your supply chain, and is usually implemented on a platform that forces you to relent your control.

See you in the fallout shelter.

That Harsh World We Call Reality

This is pretty funny. And it is also very common.

Some technical people can hardly imagine this, but there is an entire universe around production applications. We, as an industry, dismiss that universe as ‘the end-users’ or ‘the business’. We invent labels,  processes and documents, like SOA, stakeholders, subject matter experts, specs and SCRUM, to shield us as much as possible from that harsh world we call reality. 

Much like a house, an enterprise application is a clean, pristine construction. All foundations in place, freshly painted and not spoiled by that harsh world we call reality. Afterwards, actual people move in (imagine that) and start to fill it up with stuff, repaint random rooms, replace the light fixtures, have a party and kick some holes in the dry wall. And they keep doing that for years: hiring different handy men, following different decoration trends and subletting it to those crazy college kids who completely wrecked the basement.

Enterprise applications follow the same life-cycle: trends, developers and users change of the years, all adding cruft and inconsistency to the application, but more importantly: mixing and entrenching it in the real world it exists in. This is not necessarily a bad thing either: the application were made to function by its users, either by changing the application itself, or by adding strange processes around them. 

TLM is MAKE’s process of modernizing legacy applications which have been in production for usually 20 years or so. Often, they are the real estate equivalent of a crack-house: completely integrated with thatharsh world we call reality. Re-architecting a new application allows our customers to start from a blank slate, but has limited influence on its environment. To a certain degree, we are placing a new application in a legacy environment. Not taking this in account is a recipe for failure. TLM is not a recipe for failure (we actually sell it as a recipe for success!) and goes to quite some lengths to really understand what the meaning of the legacy system in its specific ecosystem of users, developers and other systems.

I am in the process of writing another post on how we do that and the kinds of things we have found. You know, in our very own harsh world we call reality.

- Mik Lernout

COBOL Architecture Astronauts: Mountain MOVErs?

This is another entry in my ‘Legacy development paradigms are the greatest source of legacy problems’  series.

Architecture Astronauts are developers who create a complicated solution to a simple problem in order solve a bunch of other potential, and probably imaginary, future problems. Every day developers all over the world fight their inner Architecture Astronaut, and every day, thousands of them lose that fight.

Over the last couple of years I have seen a lot of legacy systems built in COBOL, NATURAL, RPG and PL/1.* Expecting overly designed hyper-abstract “frameworks”, I am always surprised at the incredible pragmatism and restraint shown by the developers of those systems. At worst you’ll find a dynamic call, where the name of the program being called is passed in as a variable, or a common error routine. 

Legacy mainframe code usually is incredibly verbose, not necessarily because the language requires you to write a lot, but because legacy developers did not reuse business logic or at the very least they try to avoid doing it. Or: 20 years ago code reuse was the stuff Architecture Astronauts came up with while mainstream mainframe developers avoided it.

Programmer A: So, I need to validate this postal code. Didn’t you already write code that does that?

Programmer B: Oh yeah, here: copy & paste the bugger!

Programmer A: Eh: can’t we move that in a common function?

Programmer B: ARCHITECTURE ASTRONAUT! ARCHITECTURE ASTRONAUT!

As usual I am exaggerating, but there is a core of truth there: reuse was not harder to implement than in modern languages, but developers just did not do it on a regular basis. I’d like to say that that was caused by unsophisticated specification and design phases and a less mature software engineering practice, but that is not true: most, not all, reuse in modernized development is done by developers, while implementing, after the designs were created.

Developers communicate business needs in code, and computer restrictions into conversations with users. But the whole practice of software development itself is also guided by a higher level of fads: assumptions about what can, should and may be done in code and how it should be implemented. The more I think about it, the more important these ‘paradigms’ seem to be compared to the individual choices of each developer or team. The quality and business alignment of your system is heavily influenced by the paradigm that was used to develop it.

Next in this series: ‘SOA: where Architecture Astronauts mate’.

- Mik Lernout

* Note that I always refer to legacy languages in uppercase but I do not line break after 80 characters. Inconsistent, I know.