Why Does OS-COMMAND Hide Shell Script Failures in Progress OpenEdge?
OS-COMMAND hides failures because it does not return the script's exit code and discards everything the script writes to standard error (stderr). If a script fails halfway because of a full disk or missing permissions, ABL simply moves on to the next line of code as if nothing happened.
A typical call looks like this:
OS-COMMAND VALUE("myscript.sh").
An exit code is the number a program returns when it finishes: 0 means success, and anything else signals a failure. Stderr is the channel most scripts use for error messages. Without either of them, a failed backup looks exactly like a successful one. The Progress Community article on OS-COMMAND, shell scripts and PASOE explains more about how the statement behaves.
How to Capture Exit Codes and Errors with INPUT THROUGH
The fix is to replace OS-COMMAND with INPUT THROUGH, which lets ABL read a command's output line by line. To make it reliable, merge stderr into the normal output and print the exit code as a tagged line after the script finishes. ABL can then read both the error messages and the real result.
Progress also covers this question in its guide on how to capture the return value from a shell script. Two things need to happen:
- Merge stderr into standard output (stdout) with the shell redirection 2>&1, so error messages are not lost.
- Print the exit code straight after the script finishes, tagged with a marker that will not be confused with real output. This carries the exit code, which only the shell knows, back into the stream ABL is reading.
How does the shell command work?
The full shell command wraps the script, redirects its errors and prints the exit code in one line:
/bin/sh -c "myscript.sh 2>&1 ; echo @@EXIT_CODE=$?"
- /bin/sh -c "..." starts a shell and runs everything inside the quotes as one command line. This is needed because 2>&1, ; and echo are shell syntax that ABL does not understand on its own,
- myscript.sh is the script being run,
- 2>&1 merges file descriptor 2 (stderr) into file descriptor 1 (stdout), so error messages arrive in the same stream as normal output,
- ; runs the next command whatever happens, so echo always runs, whether the script succeeded or failed,
- echo @@EXIT_CODE=$? prints one more line after the script finishes. $? holds the exit code of the last command, and @@EXIT_CODE= is a made-up marker that is unlikely to appear in the script's own output.
What does the sample backup script do?
The sample script, sampleScript.sh, archives a data directory with tar. If tar fails, the script prints an error to stderr and exits with code 10. Any non-zero number would work, and 10 is only an example.
#!/usr/bin/env bash
set -euo pipefail
BACKUP_DIR="/opt/app/backups"
SOURCE_DIR="/opt/app/data"
LOGFILE="/var/log/app/backup.log"
mkdir -p "$BACKUP_DIR"
echo "[$(date '+%F %T')] Starting nightly backup of $SOURCE_DIR"
ARCHIVE="$BACKUP_DIR/data-$(date '+%Y%m%d').tar.gz"
if tar -czf "$ARCHIVE" -C "$SOURCE_DIR" . ; then
echo "[$(date '+%F %T')] Archive written to $ARCHIVE"
else
echo "ERROR: tar failed while archiving $SOURCE_DIR" >&2
exit 10
fi
echo "[$(date '+%F %T')] Pruning old archives, keeping 7 copies"
ls -1t "$BACKUP_DIR"/data-*.tar.gz | tail -n +8 | xargs -r rm -f || true
echo "[$(date '+%F %T')] Backup finished successfully" | tee -a "$LOGFILE"
exit 0
How does the runScript.p wrapper read the output?
The runScript.p procedure takes a command, runs it through INPUT THROUGH and returns the exit code together with everything the command printed. If the exit code is not 0, it raises an ABL error instead of returning quietly.
DEFINE INPUT PARAMETER pcScript AS CHARACTER NO-UNDO. /* full command */
DEFINE OUTPUT PARAMETER piExitCode AS INTEGER NO-UNDO.
DEFINE OUTPUT PARAMETER pcOutput AS LONGCHAR NO-UNDO.
DEFINE VARIABLE cCmdLine AS CHARACTER NO-UNDO.
DEFINE VARIABLE cLine AS CHARACTER NO-UNDO FORMAT "x(10000)".
DEFINE VARIABLE cSentinel AS CHARACTER NO-UNDO INITIAL "@@EXIT_CODE=".
DEFINE VARIABLE cShellPath AS CHARACTER NO-UNDO.
cShellPath = OS-GETENV("OE_SHELL_PATH").
IF cShellPath = ? OR cShellPath = "" THEN
cShellPath = "/bin/sh".
cCmdLine = cShellPath + ' -c "' + pcScript + ' 2>&1 ; echo ' + cSentinel + '$? ' + '"'.
INPUT THROUGH VALUE(cCmdLine) NO-ECHO UNBUFFERED.
REPEAT ON ENDKEY UNDO, LEAVE:
IMPORT UNFORMATTED cLine.
IF cLine BEGINS cSentinel THEN DO:
piExitCode = INTEGER(SUBSTRING(cLine, LENGTH(cSentinel) + 1)).
NEXT.
END.
pcOutput = pcOutput + cLine + CHR(10).
END.
INPUT CLOSE.
IF piExitCode NE 0 THEN
RETURN ERROR SUBSTITUTE("Shell command failed (exit &1)", piExitCode).
Here is what each part does:
- pcScript is the command to run, for example "./sampleScript.sh --mode nightly". piExitCode and pcOutput are filled in as the procedure runs,
- cShellPath chooses the shell. Unless it is overridden, it defaults to /bin/sh,
- cCmdLine builds the full command: <shell> -c "<script> 2>&1 ; echo <marker>$?",
- INPUT THROUGH VALUE(cCmdLine) NO-ECHO UNBUFFERED starts the command and treats its output as a readable stream, the same way INPUT FROM treats a file. NO-ECHO stops ABL from printing the output to the screen, and UNBUFFERED reads each line as soon as it is available,
- REPEAT ON ENDKEY UNDO, LEAVE reads one line at a time and stops cleanly when the input ends, instead of raising an error,
- IMPORT UNFORMATTED cLine reads exactly one line,
- IF cLine BEGINS cSentinel checks for the marker line. If it is found, SUBSTRING() takes the number after the marker, INTEGER() converts it, and the result is stored in piExitCode. NEXT skips the line, so the marker never reaches the caller,
- pcOutput = pcOutput + cLine + CHR(10) adds every other line to the transcript. CHR(10) adds back the line break that IMPORT UNFORMATTED removes,
- INPUT CLOSE closes the stream, and the final check raises an ABL error if the exit code is not 0.
Two details to keep in mind:
- FORMAT "x(10000)" caps how many characters are read per line. Raise it if you expect longer lines,
- OE_SHELL_PATH exists only for testing outside a real Linux host. Leave it unset in production, so the procedure uses /bin/sh.
How do you call the wrapper from ABL?
The doBackup.p procedure calls runScript.p and logs the captured output whether the script succeeds or fails. If the exit code is not 0, it also logs the failure and raises an error.
DEFINE VARIABLE iRC AS INTEGER NO-UNDO.
DEFINE VARIABLE lcOut AS LONGCHAR NO-UNDO.
RUN runScript.p (INPUT "./sampleScript.sh --mode nightly", OUTPUT iRC, OUTPUT lcOut).
LOG-MANAGER:WRITE-MESSAGE(STRING(lcOut)).
IF iRC NE 0 THEN
DO:
LOG-MANAGER:WRITE-MESSAGE("Backup FAILED, exit code " + STRING(iRC)).
RETURN ERROR "Nightly backup failed — see log for captured output.".
END.
I ran the test in Git Bash. The first screenshot shows a successful run:
To force a failure, I removed the source directory before running the script:
With plain OS-COMMAND, both runs would have looked identical, and the failure would have gone unnoticed.
How to Handle Large Progress OpenEdge Log Files Without Overloading the AVM
Keep large log files out of ABL memory. Use logrotate with the copytruncate option to rotate Progress OpenEdge logs without restarting the broker, and use awk to filter them at the operating system level. ABL then only receives the few lines it actually needs.
Reading a log with INPUT FROM works well for small files. With multi-gigabyte logs, however, it can exhaust memory and slow down an otherwise healthy database broker.
Why does the OpenEdge broker need copytruncate?
The broker, which is the database server process, keeps its log file open for as long as it runs. If a tool renames or deletes the file during rotation, the broker keeps writing to the old file, and new log lines are silently lost.
The copytruncate option in logrotate avoids this. It copies the current contents to a new file, then empties the original file in place. The broker keeps writing to the same file without a restart, and no log lines are lost. Progress describes the same issue in 'How to truncate UBROKER log files while the Broker is running on UNIX', and the Progress documentation covers archiving and truncating database logs.
How do you configure logrotate for OpenEdge logs?
logrotate is a small Linux utility, usually installed by default, that renames, compresses and removes old log files. It ships with rules for common system logs but knows nothing about /var/log/openedge/, so OpenEdge logs keep growing until you add a config for them:
/var/log/openedge/broker.log
/var/log/openedge/client*.log {
daily
rotate 14
maxsize 500M
compress
delaycompress
missingok
notifempty
copytruncate
}
Save this file as /etc/logrotate.d/openedge. It tells logrotate to rotate the logs daily, or earlier if they exceed 500 MB, keep 14 old copies and use copytruncate so the broker is never disrupted.
The two lines before the { list the files the rules apply to. /var/log/openedge/broker.log names one exact file, while /var/log/openedge/client*.log matches client1.log, client2.log and so on. The rules inside { } apply to all of them.
How do you filter log errors with awk from ABL?
The extractLogErrors.p procedure builds an awk command that filters a log by date and the word ERROR, then runs it through the same runScript.p wrapper. The caller receives an exit code and the matching lines in the same way, whatever log file or date is passed in.
DEFINE INPUT PARAMETER pcLogFile AS CHARACTER NO-UNDO.
DEFINE INPUT PARAMETER pcFilterDate AS CHARACTER NO-UNDO.
DEFINE VARIABLE iRC AS INTEGER NO-UNDO.
DEFINE VARIABLE lcErrors AS LONGCHAR NO-UNDO.
DEFINE VARIABLE cAwkCmd AS CHARACTER NO-UNDO.
cAwkCmd = SUBSTITUTE("awk '/^&1/ &&&& /ERROR/ ~{ print ~}' &2", pcFilterDate, pcLogFile).
RUN runScript.p
( INPUT cAwkCmd
, OUTPUT iRC
, OUTPUT lcErrors ).
IF iRC NE 0 THEN
RETURN ERROR SUBSTITUTE("Log filtering failed (exit &1)", iRC).
IF lcErrors = "" THEN
LOG-MANAGER:WRITE-MESSAGE(SUBSTITUTE("No ERROR entries found for &1 in &2", pcFilterDate, pcLogFile)).
ELSE
LOG-MANAGER:WRITE-MESSAGE(SUBSTITUTE("ERROR entries for &1:~n&2", pcFilterDate, lcErrors)).Here is how it works:
- pcLogFile and pcFilterDate set the log file to search and the date to filter on, for example "2026-09-19".
- cAwkCmd builds the awk command from a template.
- RUN runScript.p runs the command. The wrapper does not care whether it runs awk or a backup script. It returns the exit code in iRC and the output of awk in lcErrors.
- If iRC is not 0, for example because of bad syntax or a missing file, the procedure raises an error. This is the same fail-fast pattern as in doBackup.p.
- Only then does it check lcErrors. If it is empty, awk found nothing for that date, and a "No ERROR entries found" message is logged. Otherwise, the matching lines are logged.
lcErrors is a generic output variable, like lcOut in doBackup.p. It only contains ERROR lines because that is what the awk command prints. The results are written to the OpenEdge client log through LOG-MANAGER, so the source log file is never modified.
Two parts of the template look unusual because of ABL's own string rules:
- ~{ ~} escapes the braces. Without the tildes, the ABL preprocessor reads { print } as an include-file reference.
- &&&& produces &&. SUBSTITUTE() treats & as the start of a placeholder (&1, &2), so the && operator in awk has to be written twice.
How do you test the log filter?
To test the filter, I used this sample log:
2026-09-18 08:12:03 INFO Broker started, listening on port 5162
2026-09-18 09:44:11 INFO Client session 1042 connected
2026-09-18 14:02:57 ERROR Database connection pool exhausted, retrying
2026-09-18 22:10:45 INFO Nightly batch job completed
2026-09-19 00:01:02 INFO Broker health check OK
2026-09-19 03:15:29 ERROR Disk usage on /db01 exceeded 90% threshold
2026-09-19 07:30:10 INFO Client session 1043 connected
2026-09-19 07:45:22 ERROR Timeout waiting for record lock on customer table
2026-09-19 12:00:00 INFO Broker health check OK
2026-09-19 18:20:38 ERROR Backup script exited with non-zero status
The test runs in doLogCheck.p, a thin caller built on the same idea as doBackup.p. It reads the date passed with the -param startup option through SESSION:PARAMETER. If nothing is passed, it uses today's date in YYYY-MM-DD format, which matches the log's timestamps, because the awk pattern looks for the date at the start of each line.
DEFINE VARIABLE cDate AS CHARACTER NO-UNDO.
cDate = SESSION:PARAMETER.
IF cDate = ? OR cDate = "" THEN
cDate = STRING(YEAR(TODAY), "9999") + "-" + STRING(MONTH(TODAY), "99") + "-" + STRING(DAY(TODAY), "99").
RUN extractLogErrors.p (INPUT "sampleAppLog.log", INPUT cDate).
The output looks like this:
For very large files, such as logs measured in tens of gigabytes, do not run this from the broker or from a session serving live users. Run it as a separate batch job that writes its results somewhere ABL can read later. That way, waiting for awk never holds up anything user-facing.
How to Set Up This Pattern in Progress OpenEdge
Setting up the pattern takes six steps: make the script executable, call it by a path the shell can find, deploy the ABL wrapper, enable the OpenEdge client log, install the logrotate config and test both success and failure.
1. Make the script executable and control who can run it.
Run chmod +x sampleScript.sh, because a file that is not executable will not run. Limit permissions so only the OpenEdge account can run the script directly, and route everyone else through the ABL wrapper so their runs are logged too.
2. Use a path the shell can find.
A bare file name like sampleScript.sh only works if its folder is on PATH. Use ./sampleScript.sh for the same folder, or a full path otherwise.
3. Set up the ABL side.
Put runScript.p where other .p files can call it, and call it the same way doBackup.p and extractLogErrors.p do.
4. Turn on the OpenEdge client log.
Before calling LOG-MANAGER:WRITE-MESSAGE, enable logging with the -clientlog startup option or by setting LOG-MANAGER:LOGFILE-NAME in code. Otherwise, the messages have nowhere to go.
5. Install the logrotate config.
Place it in /etc/logrotate.d/ on the Linux host. Before trusting it, run logrotate -d /etc/logrotate.d/openedge. The -d option shows what would happen without touching any files.
6. Test both outcomes.
Run the job once when everything works, then break it on purpose, for example by pointing the backup script at a folder that does not exist. Confirm that the exit code and error text appear in the log. Until you have seen a failure captured, the failure handling is not proven.
What Challenges Should You Expect?
Most problems come from ABL's string handling, data type limits and differences between environments. The table below lists the issues found during testing and how to fix them.
What Did the Tests Show?
Both patterns worked as intended. The shell logic was tested in Git Bash on Windows rather than on a Linux host. Git Bash provides a genuine POSIX shell (/bin/sh, awk, chmod), so the shell and awk logic behaves the same way as on Linux. The ABL code ran under a real AVM.
- Use case 1: the successful run returned iRC = 0 with the full transcript. The forced failure returned iRC = 10 together with the real tar error line from stderr.
- Use case 2: the awk filter returned exactly the three ERROR lines for 2026-09-19 from sampleAppLog.log, and correctly returned nothing for a date with no matches.
- runScript.p: after the LONGCHAR and IMPORT UNFORMATTED fixes, it compiled and ran cleanly under a real AVM. This confirms that exit code capture works at the ABL level, whatever command the wrapper runs.
Key Takeaways for Running Shell Utilities from Progress OpenEdge
Calling the shell from Progress OpenEdge does not have to mean losing visibility. Running scripts through INPUT THROUGH with merged stderr and a marker line turns silent failures into exit codes and error text that ABL can check and log. Moving log filtering to awk and log rotation to logrotate with copytruncate keeps large files out of AVM memory and keeps the broker online.
These patterns solve three practical problems:
- Silent failures become visible. OS-COMMAND always looks successful to ABL. INPUT THROUGH with a marker line gives ABL the real exit code and error text.
- Log size stops being an ABL memory problem. Filtering happens in awk, outside the AVM, and ABL only receives the matching lines.
- Rotation no longer requires a broker restart. copytruncate empties the log file in place, so the broker's open file stays valid.
If you're planning to improve the reliability of your Progress OpenEdge applications or need help with scripts and log management, our Progress OpenEdge team is here to help. Book a consultation with our specialists to discuss your setup.
