EDISeparator Adapter for Routing,Grouping and Splitting of XML messages.

SAP Middleware solutions were already known for the capabilities to support varying Application-to-Application (A2A) and where B2B integration was needed, third party solution was typically the option. But since the launch of SAP B2B Adapters, the choice of using SAP PI/PO/PRO as a middleware solution has become inevitable, because of its capabilities to support A2A, B2B as well on premise and cloud integrations. The options available to achieve splitting one message into several ones, include:

• ccBPM (only relevant for dual stack deployment)
• SAP Business Process Management tools (BPM)
• SAP PI/PO/PRO tools (e.g with B2B Adapters; which is the option that has been used in this scenario).

EDI Separator is not only capable of splitting EDI messages but also we can leverage the same functionality for XML based on any field at any level of hierarchy. With the help of B2B EDI Separator Adapter we can do the routing/splitting and grouping of the XML messages.

Figure: SAP B2B EDI Separator adapter in use for message split.

Scenario Description:

According to the business requirement, this is a B2B scenario that takes advantage of the SAP B2B EDI separator adapter in order to meet all demands. Below is the sample POC

Step 1: Employee Data with multiple records is sent in one big message from System A to the SAP PRO environment. There, the message is processed by an XSLT mapping to sort the employee data per country (e.g. if there are 10 employees from 4 different countries, all will be sorted into 4 countries), and subsequently an EDI Separator Receiver channel will receive the sorted message for onward processing.

Step 2: The EDI Separator Receiver channel triggers an EDI Separator Sender channel based on configured parameters. Next, it does the splitting of this message into multiple xml messages based on a defined xpath expression (xpath expression to locate the country tag and split into multiple messages based on the number of sorted and grouped countries). Subsequently, it will be sent out to the receiving system, System B, which will be done by FTP.

 

Figure: Configuration Overview for the use of an EDI Separator Sender and Receiver Channels

Split mapping

As described above, the sender system – system A – parses 1 large message to the SAP PRO environment. This message contains employee data, such as first names, last names, address data, function, join date, and so forth. All these elements are present in the message, and are hierarchically listed per employee. A message can consist of multiple employee records, and are randomly put into the message which means there is no sorting done per employee name, country code, or whatsoever.

In this example, the requirements for the mapping logic differ per country + the receiving party demands for separate files per country. Two very good reasons to realize a message split in SAP Process Orchestration!

Sorting the message

Part of the message structure from System A is an element that is always occurring per employee record, and is reflecting the country code of the country where the employee is working. Based on this field it is possible to sort the message per country, meaning the Country Code will act as a key. An Intermediate XSD (called XSD EDI Separator in below screenshot) is created, based on the XSD from System A, where another hierarchy level has been added and which is called ‘Country’.

Under the newly created CountryX node, the employees that work in a particular country are listed. A new Country node will be created as soon as the input message contains a worker from a Country which so far was not present in the target message. Please keep in mind that the message will not be split yet in this mapping; the mapping only facilitates the splitting. This split part will be done later by the EDI Separator Component itself.

XSLT Mapping

Right now the focus will be on how to create a mapping that sorts the single large message in such a way that the EDI Separator component is able to split the message per Country Code. Again, this splitting will later on be done based on the Country node.

In order to create the intermediate message structure, a SAP PRO consultant has three options: A graphical mapping, a XSLT mapping and a JAVA mapping.

A graphical mapping should be fairly easy with the standard sort and ‘sortByKey’ statements that are available. Next, there is the possibility to create User Defined Functions where needed. However, when this option is chosen one comes across (at least) two difficulties:

• All individual elements should be sorted by the key you are using – in this case Country Code. In case of a XSD with numerous elements, this copy pasting of logic becomes quite a chore.
• In case segments can occur just once, multiple times, or not at all, it is likely that one comes across context issues. For instance, how to make sure the right amount of context changes are still present under the correct employee?

Mapping Logic

During the second step a new message structure needs to be defined; the one that is called Intermediate Message in this blog. A variable has been created first, variable ‘currentCountry’, and will be used to generate the new message node ‘Country’. The for-each statement creates a key based on all the Country IDs. This key is then passed to variable currentCountry.

Finally, all employee data records and all hierarchical segments belonging to the specific Country Code will be copied 1:1 under node Country. The same happens when a new Country Code appears from the source message; all employee records that have that exact same Country Code will be copied under the newly created Country node.

ID Configuration:

Integration Configuration 1:

First Integrated Configuration is just a regular setup which receives messages by any means(FTP/SOAP/JDBC/Idoc etc). Without any further conditions all of these messages, sent by Source System, will be forwarded to the EDI Separator component – in this case named INTEGRATION_ENGINE_JAVA_PO1.

ICO # 1 For the first part of message processing.

In the Receiver Interfaces tab, add a XSLT mapping before it gets routed towards the EDI Separator. Here the XSLT mapping, as described above, can be incorporated in order to sort the data structure and prepare the message to be split by the EDI Separator component. We will use a pass through scenario here.

ICO # 1 For XSLT mapping

ICO # 1 EDI Separator Receiver Channel

Configuration Details for EDI Separator Receiver Channel.

ICO #2 EDI Separator Sender Channel for Routing the message from EDI Separator Receiver to target System.

In this ICO you can use your actual mapping and routing conditions. Individual messages are executed one by one on second ICO and files will be sent to the target System.

File Lookup on File Server Using the FTP Credentials

There could be requirement when we have to do a file lookup on different server. My previous code will work when we have server mounted over our PI/PO system. From below code snippet you can do a lookup on any FTP server with the help of credentials.

Below is the code:

String host = "<hostname>";

int port = 21;

String username = "anonymous";

String password = "pwd123";

String str = "";

FTPClient ftpClient = new FTPClient();

try {

    ftpClient.connect(host, port);

    ftpClient.login(username, password);

    ftpClient.enterLocalPassiveMode();

 

    String remoteDirPath = "/sap/PO1/PO_Test/Training/Rahul/Input";

 

   FTPFile[] files = ftpClient.listFiles(remoteDirPath);

 if (files != null && files.length > 0) {

            for (FTPFile aFile : files) {

                str = aFile.getName();

result.addValue(str);

// str = str + currentFileName;

}

      }

result.addValue( "Connection Successful");

    ftpClient.logout();

    ftpClient.disconnect();

} catch (IOException ex) {

    System.err.println(ex);

}

This code can be used for anonymous login also. If you are doing anonymous login on FTP server then password string must be blank.


If you are liking this blog then do provide your feedback.

Using File Lookup in SAP PI/PO

PI has two standard types of lookup available in ESR (i.e. JDBC and RFC lookup).  But we can achieve two more types of lookup in PI to fulfill our needs, those are File and SOAP lookup.

File Lookup is done when we want to map a value against an incoming payload value. In this scenario file may be maintained on some other server. We do a lookup to read the value against incoming field and map that value to the target payload.

To achieve this functionality we need to write a small UDF, which will leverage our requirement.

Step1: Create a UDF and import two packages as mentioned in screenshot.

           java.net.URL

           java.net.URLConnection

Step2: Copy paste below code in your PDF

String custId = "";
try{

URL url =new URL("ftp://<UserName>:<Password>@<IPAddress>/ <pathtodirectory>/ <filename.txt>");

URLConnection con = url.openConnection();
InputStream lookupStream = con.getInputStream();

InputStreamReader reader = new InputStreamReader(lookupStream);
BufferedReader buffer = new BufferedReader(reader);

String read;
while((read=buffer.readLine()) != null){
String temp = read.substring(0,key.length());
if(key.equals(temp)){
custId = read.substring(key.length()+1,read.length());

if (  read.substring(key.length()+1,read.length()) != "00"){

int num = Integer.parseInt( read.substring(key.length()+1,read.length()));
num = num+2;

custId = Integer.toString(num);
}
}
}

buffer.close();

} catch (Exception e){
return e.toString();
}

return custId;

Step3: Create a file on external server which has input and output separated by "=".

Step4: We can verify our code through Display queue by giving input in the source field.
Below screenshot shows we are getting output as expected.

This is the simplest way of doing a file lookup in SAP PI through UDF.