EDI Conversion File Polling Service in Java
EDI Conversion Service Overview
This is a sample Java service which scans a directory, and upon seeing new EDI files, converts them into XML files. There is a corresponding .Net version of this program also.
EDI Service Details
EDI files are often dropped into a directory, with the intention that some program will iterate through them and convert them. It's possible that they were placed here by another program across a network share or via FTP, or another application on this same directory.
This service polls a directory, whose name is given on the command line, every 15 seconds. If it sees a file whose name ends in ".edi" that hasn't been converted into a corresponding ".xml" file, it does so.
If there are any errors, the partial ".xml" file is removed, and instead a file with the suffix ".bad" is created, containing the text of the error.
The program will terminate if a file called STOP is created in the scan directory.
A log of the actions is written to the console. That log might look something like this:
2007-12-27T17:02:14 Scanning c:\inbox 2007-12-27T17:02:14 Found 4 files 2007-12-27T17:02:14 Converting 00203394.edi 2007-12-27T17:02:15 Converted 00203394.edi 2007-12-27T17:02:15 Converting 831.edi 2007-12-27T17:02:15 Converted 831.edi 2007-12-27T17:02:15 Converting order.edi 2007-12-27T17:02:15 Error on order.edi 2007-12-27T17:02:15 Converting sx.edi 2007-12-27T17:02:15 Converted sx.edi 2007-12-27T17:02:15 Sleeping 2007-12-27T17:02:30 Scanning c:\inbox 2007-12-27T17:02:14 Found 4 files 2007-12-27T17:02:30 Sleeping
EDI File Polling Service Implementation in Java
First, set up the package, imports, and initialization code.
package com.ddtek.tutorial; import java.io.*; import java.text.SimpleDateFormat; import java.util.Date; import javax.xml.transform.stream.*; import com.ddtek.xmlconverter.*; public class Scanner { static public void main(String[] args) { new Scanner(args[0]); }
Also, a small utility function to return the date/time for the logging output.
static private SimpleDateFormat m_sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss"); static private String now() { return m_sdf.format(new Date()); }
Java doesn't have a way to natively poll for changes in a filesystem structure (at least not until something like NIO.2 passes to, among other things, add watch events to Java), so we need to periodically poll and look for changes.
Since we'll need one later, we instantiate the ConverterFactory. This same one will be used throughout the execution of the program.
We look for the presence of a file called STOP. If we see that, we just quit. This provides a clean way to exit the application.
Scanner(String dir) { ConverterFactory cf = new ConverterFactory(); for (;;) { System.out.println(now() + " Scanning " + dir); File fend = new File(dir + "/STOP"); if (fend.exists()) { System.out.println(now() + " Stopped " + dir); break; }
Next, the task is to identify all of the files ending in ".edi". We use a custom FilenameFilter anonymous class.
File fdir = new File(dir + '/'); String list[] = fdir.list(new FilenameFilter() { public boolean accept(File directory, String name) { return name.toLowerCase().endsWith(".edi"); } }); System.out.println(now() + " Found " + list.length + " files");
As we run through the list of potential candidates, we weed out those that have already been converted, or that have failed to convert.
for (int i = 0; i < list.length; i++) { File fedi = new File(dir + '/' + list[i]); File fxml = new File(dir + '/' + list[i].substring (0, list[i].length() - 4) + ".xml"); if (fxml.exists()) { //System.out.println(now() + " Already handled " + list[i]); continue; } File fbad = new File(dir + '/' + list[i].substring (0, list[i].length() - 4) + ".bad"); if (fbad.exists()) { //System.out.println(now() + " Found previous error in " + list[i]); continue; }
Now the task is to convert one. Notice how small the actual "convert" code is, relative to the size of the entire service!
We make sure we store the FileOutputStream in a variable that is scoped outside of the try, so that if there is a failure we can close the ".xml" file it points to and remove it.
System.out.println(now() + " Converting " + list[i]); OutputStream os = null; try { ConvertToXML c2x = cf.newConvertToXML("EDI"); os = new FileOutputStream(fxml); c2x.convert(new StreamSource(fedi), new StreamResult(os)); os.close(); System.out.println(now() + " Converted " + list[i]);
When things go sour, we remove the ".xml" file and record the contents of the exception thrown in a ".bad" file.
} catch (Exception e) { System.out.println(now() + " Error on " + list[i]); try { os.close(); fxml.delete(); } catch (Throwable t) {} try { os = new FileOutputStream(fbad); PrintWriter pw = new PrintWriter (new OutputStreamWriter(os, "utf-8")); e.printStackTrace(pw); pw.flush(); os.close(); } catch (IOException ioe) {} }
Finally the program can sleep until the next scanning operation.
}
System.out.println(now() + " Sleeping");
try {
Thread.sleep(15000);
} catch (InterruptedException ie) {}
}
}
}
Improvements
Of course, this is only sample code to get you started. Some areas for improvement might be:
- Pass the polling duration on the command-line
- Look for STOP in another directory that maybe isn't network-mounted, to keep a user from stopping the service by uploading a file called STOP
- Using a native callout via JNI to get real directory change notifications
- Putting the output files in different directory
- Moving the input files to a different place after processing
- Allowing different URI "EDI" options, perhaps based on filename
- Automatically generate acknowledgements: ACK for HL7, 997 for X12 or CONTRL for EDIFACT/EANCOM/IATA. DataDirect XQuery is ideal for generating these
- Process the XML using DataDirect XQuery is ideal for generating these
Java XML Converters as Components
The source code can be downloaded from edi_polling_java.zip.
The important point to remember is that DataDirect XML Converters are components that can be hooked into your applications wherever needed. Try them out and see how they can simplify management of complex data by handling conversion and validation for you. Download XML Converters today!
