Unlock Your RPG Applications to Enable Modern Online Experiences

By Dan Magid

What programming languages or frameworks do you think of when you hear web development? Probably JavaScript; maybe PHP or Python. But RPG? Not likely.

Nonetheless, for those who rely on longstanding IBM i systems to support their mission-critical operations, accessing current RPG applications is a necessity for powering fully functional, modern mobile applications. Of course, you might ask how a 60-year-old programming language could possibly handle the demands of modern online interactions.

RPG wasn’t originally built to power the web, but believe it or not, it may just be your fastest, most cost-effective option. With technological advances, it’s not always about what something was decades ago, but rather what it can do for you today.

In fact, RPG-based applications are being repurposed in ways their original developers could not have imagined, with amazing results. There is a significant amount of domain knowledge about the unique way your company does business built into your RPG applications. With the rise of the API economy, developers need that functionality for web, mobile and other applications. While RPG itself was not built with the web in mind, evolutionary technology has turned RPG into a web powerhouse, enabling the creation of web and mobile UIs—and even completely reengineered workflows—using the capabilities of existing RPG applications. And, RPG programs can call APIs from virtually any other application. Now, RPG code from 40 years ago underpins some of the most innovative applications you’ll see for call center and customer self-service applications, for example.

But why go this route when you could replace your old applications with pre-packaged or custom alternatives? The answer is that building upon what you already have—your venerable RPG applications— is faster, lower risk and significantly less expensive than replacing an entire system. Remember that those applications often have decades of business rules that must be replicated by any new system you buy or build from scratch, and that the TCO of your IBM i is on average about 1/3 of the most popular web and mobile platforms.* By keeping the back-end RPG application in place and using APIs or modern UIs, businesses can deliver renewed value from their current investments while still offering the kinds of features and experiences employees and customers need.


* Quark + Lepton, 2017 Study

Guest Blogger

Daniel Magid is Managing Director of Rocket Software’s Application Lifecycle Management & DevOps lab, and is a recognized authority on helping leading organizations achieve compliance through ALM solutions and DevOps best practices. He has written a variety of articles for leading IT publications and is a regular speaker at technology conferences.

The Strange World of Bug Bounty Hunters

How do software companies find dangerous bugs in their code? Ideally, their own QA departments discover them before it’s released. Sometimes they find out only when customers have problems. That might mean after there’s been a breach. But sometimes they hear about bugs from freelancers who find them in return for a reward. These people are called bug bounty hunters.

Some companies find it worthwhile to offer payment for bug reports. Learning about security holes before anyone can exploit them can save the company’s reputation, which is worth a lot of money. Recently Google paid $112,500 to a researcher for discovering a flaw that could have let a website push arbitrary code into an Android device. Having to deal with it after criminals found out could have been far more expensive.

The Mind of the Bounty Hunter

Bug hunting makes up half or more of some people’s income. They spend hours every day looking for flaws in websites. How different are they, really, from those who do the same thing and use their discoveries to steal information? Sometimes the same person plays both sides of the fence, depending on which one is paying better.

It’s the challenge, perhaps even more than the money, which motivates them. Anyone with those skills could get a well-paying job in QA. But they’d rather be on their own, chasing down bugs without reporting to a boss. Their attitude is, “So you think I can’t break this code? I’ll show you!”

The Benefit to Users

A bounty may encourage hackers to stay within the law. It can even motivate them to work harder at what they like to do. It’s easier to explain their income when it comes from Google rather than the Shadow Brokers, and there’s less chance of blackmail afterward. When they report bugs, the software publisher can fix them before anyone is harmed.

Software bounty hunters are a strange breed, there’s no question. But they do all of us some good.

Need to learn more about IT security? Take a look at POWERUp18 security sessions.

A Powerful Way to Run Unix and Open Source Tools from a Program

Perhaps the biggest area of growth in IBM i programming over the past several years has been the Open Source languages. There are thousands of utilities, mostly designed for Unix, that you can run in the QShell and PASE environments, and these have become very popular on IBM i! However, running these tools from your RPG and CL programs can be tricky. This post will introduce you to a free utility called UNIXCMD that makes it much easier.

Why is it Tricky?

It’s tricky because there’s a difference in the way IBM i and Unix systems run programs. When a program is called on Unix, a new “process” is created, this is very much like a new job on IBM i, except that it is created each time a program is called. This gives the calling program a choice, it can either stop and wait for the called program to finish, or it can continue and run simultaneously with the program it called. In a way, this is similar to submitting a batch job on i, but it’s different in that both programs can interact with the user’s display.

There are two common ways of running Unix programs, PASE via the QP2SHELL API and QShell via the STRQSH (or it’s alias QSH) CL command.

The QP2SHELL API runs a PASE program directly in the current job, which will cause problems if the job expects to be able to run simultaneously with the caller. Programs that use multiple threads, in particular, can have strange problems that are very hard to troubleshoot. The fix is to spawn a child job to run QP2SHELL so that it’s not in the same job as the caller. To enable input and output, you need to connect pipes to that child job. This is a lot of work, and often more than a programmer bargained for. (In fact, my description is somewhat oversimplified, to keep this brief!)

The STRQSH CL command solves this by always spawning its own child job and connecting pipes to it. It does all of this for you. The problem with this is that you are limited in how you can interact with the input and output streams. For a program to work with them, the only real option is to read and write from temporary files. This works, but it cumbersome, you have to create the file, clear it, redirect the I/O, and can only read the output once the whole process has finished.

Scott’s Solution

My solution is to create tools that do the work of submitting the child job for you, and connecting the pipes to a simple interface that is easy to use from your program. For a CL program, I’ve provided simple commands that open, read, write and close the connection, allowing you to read and write in a natural way. In RPG, I’ve used the Open Access interface so that you can open the connection with RPG’s native file interface, and read and write using the standard open, read, write and close opcodes.

The RPG Interface

Let’s take a look at RPG first. My initial examples use the newer “all-free” approach to writing RPG, but if you’re unable to use newer RPG, don’t fret – see the section titled “Older RPG Code”, below.
This example switches to the /QIBM directory in the IFS, lists the files in that directory, and prints them to the spool:

Notice the HANDLER keyword on the DCL-F statement. This tells RPG to access this file through the Open Access interface. When you run any of RPG’s file opcodes against this file, it will (under the covers) call routines in the UNIXCMDOA program, that allows my tool to take control and handle all of the work for you.

The command to run is provided in the second parameter to the HANDLER keyword. Since I’m setting that command in calculations in my program, I do not open the file until the command variable is set. For that reason, the file is declared with the USROPN keyword, and I open it explicitly with the OPEN opcode.

Unix utilities allow you to run multiple commands on a single line if you separate them with a semicolon. In this example, the cd command is used to switch to the /QIBM directory, and then the ls command is run afterwards to get the output.

To get the output from the Unix ls (list directory) command, I simply use the READ opcode. Since this is a program-described file (there are never any defined fields in a Unix input or output stream) I am using RPG’s feature that lets me read program-described data into a data structure.

When I’m done, I use the CLOSE opcode to shut down the Unix process. One thing that surprises people who are new to UNIXCMD is that any errors that occur in the background Unix program will be reported on the CLOSE opcode rather than the OPEN, READ or WRITE. This is because the Unix program has the opportunity to write error messages, and will not report that it has failed until the program has ended. Catching errors can be done easily with RPG’s MONITOR and ON-ERROR opcodes.

By default, the UNIXCMD utility runs your program using the QShell interface. If you prefer to run the PASE interface and avoid the QShell environment, you can do that by prefixing the command string with “pase:”, as shown in the next example.

The examples so far have only read output from a Unix command. In the next example, I’d like to demonstrate sending data both ways. In this case, I’m calling a PHP script that calls a web service to Geocode and address. In other words, I pass an address as input, and the script returns the latitude and longitude coordinates where that address can be found. To do that, I’ve written a PHP script that receives the address from it’s “standard input” (that is, the pipe that is connected to its input stream) and write the coordinates to standard output. (If you’re interested in the PHP code, it is included in the downloadable examples on my web site.) To call it from RPG, I can simply write my data to it, and then use the READ opcode to get the results.

The first thing you’ll notice in this statement is that it sets the Unix PATH variable. This is done because many people don’t have the PHP command in their PATH. PATH is very much like a library list, except that it is a list of IFS directories that the Unix environment uses to find a command. The PATH statement adds the directory where we’ve placed the php-cli command (CLI stands for “command line interface”) so that QShell can find it.

Another important thing to note is that data sent as input through the pipe is not automatically converted from EBCDIC to ASCII or Unicode. To solve that problem, I added a call to the QShell iconv utility, which can translate between different CCSIDs. In this case, will convert between 0 (a special value that means “this job’s CCSID”) and iso-8859-1 which is CCSID 819 and is a flavor of ASCII.

A Note About Input and Output

UNIXCMD assumes that you will write all the data that is sent as input to the Unix command first, before the first time you read its output. When you read the Unix output, it will shut down the Unix input stream to signal the Unix program that no more data is coming. This works well in most applications.

However, if you would prefer that it not close the stream, this can be made with a very simple code change to the UNIXCMD utility. If this would be useful to you, please e-mail me at commonblog@scottklement.com and I’ll be glad to show you how to change it.

Older RPG Code

Sadly, not everyone has a current version of RPG. To help those people, I’ve provided a way of using UNIXCMD that’s compatible with even the oldest versions of RPG IV using the SPECIAL file interface. Here is the last example rewritten to use that approach.

Notice that the command is passed to the SPECIAL file through the PLIST. The second parameter in the PLIST is not shown in this example, but you can add a 1 character second parameter and set it to P if you want to run in the PASE environment. If you do not pass this parameter (or set it to a Q) it will run QShell instead. Here is an excerpt of code that does that:

Since the SPECIAL file approach does not require Open Access, it will work all the way back to V5R3.

Using the UNIXCMD Tool from CL

Since the CL programming language does not support Open Access, there is no way to use the standard IBM supplied SNDF or RCVF opcodes that you are used to using for a normal file. Instead, I have created my own CL commands named OPNPIPE, SNDPIPE, RCVPIPE and CLOPIPE that handle the open, send, receive and close functions, respectively. The OPNPIPE command accepts the command to run, and lets you designate whether it is PASE or QShell. Aside from these differences, the UNIXCMD utility works the same from CL as it does from RPG.

Here’s an example of listing files (like the first RPG example) using the CL interface:

Get the UNIXCMD Utility

UNIXCMD is an open source tool that is available at no charge. You can download it, the examples given in this article, and a few more examples from my web site at the following link:


The Python Programming Language

Python is a powerful, interpreted, open-source language available for most operating systems, including IBM i. It has many uses and has a very easy to learn syntax. Many packages are available to add capabilities not included in the Python core. Python is a great language to use for general purpose programming of all sorts.

Open Source - Python





Python Language Details

Python structure uses white space instead of brackets. This makes the code more readable than in similar languages such as Perl. The syntax is logical and easy to follow, even for beginners, and typing is dynamic. Even with this simple syntax, the language also has great flexibility and powerful capabilities. Object-oriented programming is possible but not required, as it is in other languages like Java or Ruby.

Web Development

One use of Python is for back-end web development. Frameworks such as Flask and Django simplify back-end development using built-in components that are popular. Rapid development and prototyping are possible using these frameworks. Python is also good for general scripting, including projects involving crawling the web or scraping information from websites.

Data Science

Python is a great language to use for data science. Useful libraries include NumPy, which adds speedy mathematical capabilities, and SciPy, which adds the ability to manipulate complex data in arrays for linear algebra and other purposes. Several other libraries are useful for doing data science in Python.


Python has a philosophy often referred to as “The Zen of Python“. This philosophy encourages simple, clean code that is easy to read and explain to others. It has several other suggestions for optimizing the use of Python to make it uniform and ideal using best practices.


A simple yet powerful open-source language, Python is a great choice for many projects. Using Python gives you many capabilities out of the box and additional packages to add more. The language is quite readable, and its uses include many technical domains.

Learn Python with COMMON’s Open Source Video Series.