Data archiving improvement notes 2018

In 2018 SAP ran an improvement project which resulted into a set of OSS notes that will make data archiving more robust and easy.

All of these notes come with manual work. Select the ones really useful.

Archiving write process improvements

Write variant maintenance has been made easier by allowing copying of variants (useful if you have many plants and company codes and want to store each one in different archive file): 2520093 – Archive administration: Enhanced variant maintenance (writing, preprocessing, and postprocessing).

To be able to detail the written file name of the archive file implement this oss note: 2637105 – Print list for archiving write jobs: Placeholders for session numbers, archive file key in title.

Archiving storage process improvements

Archiving system technical check button is available in OAC0, but not in SARA. After applying this note you can also check it in the technical settings in SARA: 2599263 – Connection test for storage systems for archiving object.

Deletion process improvements

To be able to quickly continue with interrupted archiving sessions apply this note 2520094 – Continue: Information on existence of interrupted or incomplete archiving sessions.

This note will implement checks to warn you about uncompleted previous store and delete runs: 2586921 – Run selection for deletion: Information about the existence of unstored archive files.

Some archiving object use the AIS (archiving information system) to enable the end user a quick retrieval of archiving information. This note will give warning before start of deletion if the AIS is note active for the object: 2624077 – Starting delete jobs: Check for active info structures.

Archiving overview and logging improvement

To get a better overall overview of all logs apply OSS note 2433546 – Archive administration logs: Information about errors in hierarchy display.

Direct navigation to Archive File Browser: apply OSS note 2544517 – Archive administration: Direct navigation to ArchiveFileBrowser. This note only gives you a link. You can already start the archive file browser using transaction AS_AFB:

Archive file browser


SAP database growth control: data archiving business discussions

This blog will give answers to the following questions:

  • When to start data archiving discussion with the business?
  • How to come to good retention periods?
  • What are arguments for not archiving certain data?

Data archiving discussion with the business

Unlike technical data deletion, functional data archiving cannot be done without proper business discussion and approval.

Depending on your business several aspects for data are important:

  • Auditing and Sox needs
  • Tax and legal retention periods
  • Product data requirement
  • And so on…..

Here are some rules of thumb you can use before considering to start up the business discussions about archiving:

Rule of thumb 1: the system is pretty new. At least wait 3 years to get an insight into which tables are growing fast and are worth to investigate for data archiving.
Rule of thumb 2: if your system is growing slowly, but the infrastructure capabilities grow faster: only perform technical clean up and don't even start functional data archiving.
Rule of thumb 3: if you are on HANA: check if the data aging concept for functional objects is stable enough and without bugs. Data aging does not require much work, it is only technical and it does not require much business discussions. Data retrieval from end user perspective is transparent.

Data analysis before starting the discussion

If your system is growing fast and/or you are getting performance complaints, then you need to do proper data analysis before starting any business discussion.

Start with proper analysis on the data. Use the TAANA tool to get insights into the data: how is the distribution of data per document type, per year, per plant/company code etc. If you want to propose retention period of let’s say 5 years, you can use the TAANA results to show what percentage of data you can move out of the database.

Secondly: if you have an idea on which data you want to archive, first execute a trial run on a recent production copy. There might be functional blocks that prevent you from archiving data (like not closed documents).

Third important factor is the ease of data retrieval. Some object have a nice simple data retrieval function, and some are really terrible. If the retrieval is good, the business will more easily accept a shorter retention period.

As last step you can start the business case: how much data will be saved (and how much money hence will be save) and how much performance would be gain. And how much time is needed to be invested for setting up, checking (testing!) and running the data archiving runs.

In practice data archiving business case is only present in very large systems of 5 TB and larger. This sizing tipping point changes in time as hardware gets cheaper and hourly manpower costs go up.

The discussion itself

Take must time in planning for the discussion itself. It is not uncommon that archiving discussions take over a year to complete. The better you are prepared the easier the discussion. It also helps to have a few real performance pain points to get solved via data archiving. There is normally a business owner for this pain point who can help push data archiving.

SAP database growth control: data archiving run

Questions that will be answered in this blog are:

  • Which settings do I need to make or check before data archiving run?
  • How to perform the data archiving run?
  • How to validate the data archiving run?
  • How to retrieve that archived data?

This blog assumes you have finished the basic technical data archiving setup as described in this blog.

Functional data archiving example: purchase requisitions

To explain the functional data archiving we will use Purchase Requisitions as example. Technical object name is MM_EBAN.

Start screen SARA MM_EBAN

To see which tables are archived hit the Database Tables button. Here you can see the list of tables from which data potentially be archived:

Data base tables MM_EBAN

If you want to see the other way around, which table is used in archiving objects, do put in the table as entry point, to retrieve list of archiving objects. In this example archiving objects that delete from table EBAN:

Tables that archive EBAN

Dependency of objects

By clicking the top left button on the archiving object you get the archiving dependency view. For MM_EBAN this is pretty simple: it has no dependencies.

As example for dependencies this is the overview for sales orders (SD_VBAK):

SD_VBAK dependency overview

Here you can see that before you can archive sales orders, you should archive the billing documents first. And for the billing documents, you should archive the deliveries first.

Functional archiving settings

First we have to make or check the object specific functional archiving settings.

Application specific customizing

In the case of purchase requisitions we have to set the retention periods per document type:

Set application specific residence times

Pre-processing step

Some archive object have a pre-processing step. MM_EBAN has one as well. In this step data is selected and marked for archiving (many times by setting deletion flag or other indicator).

MM_EBAN preprocessing

In the step create the variant (give it a useful name) by putting in the name and pressing Edit. On the next screen fill out your data select the log level. Go back to the first screen and select the start data and spool parameters. When both lights are green, hit the execute button. When you click the job log button you check for the results.

Example of result of pre-processing run:

Preprocessing result

As you can see not all selected data is archived. Transactions that are not completed from business point of view will not be flagged for archiving.

Write run

If you have done the pre-processing step, continue with the write step. Principle is the same: select the data and log level. Important in the write step is to correctly fill the Archiving Session Note with a useful text. This text is put as label on the archive file for later retrieval:

Archiving session note

When done plan the job and execute. Result looks like:

Write summary result

Pending on your technical system settings the file will be stored automatically or you still need to do this manually.

Deletion run

Finally we can now start the deletion run: the actual clean up of old data happens now.

Select the data files you want to archive and start the run.

Word of care with deletion: please don't select too much files and subsection in one go. Each file sub section will result into a deletion job. The deletion will put significant load on the database, since it will be pushing out a lot of data. If you are not careful you will launch easily 20 or more heavy deletion jobs that run in parallel and that might severely decrease system performance.

Result of archiving deletion run:

Deletion result

Checking archive result

The result checking is possible by looking at the technical correctness of the archive file.

In the archiving object choose the Overview button. Then select the archive file you want to inspect. A correct file should like like this:

Archive administration

In the testing phases and first production runs, you also want to do record counting. A good way is to run the TAANA transaction for key tables you want to archive before the archiving and after the archiving. The difference should match the deletion counter on the write and deletion logs. If you find differences: check for bug fix OSS notes.

Data retrieval

Retrieving archived data is different per archived object. Some retrieval is nicely integrated into the normal transaction. Some require extra transaction to run. Some retrieval is via special program.

Data retrieval of purchase requisitions can be done via SARA and choosing the read option.

Here you first need to manually select the archive files to read from (see I did not give the note and regret it, since the file has no meaning now…):

Select files for read program

Result after reading looks like this:

Read program result

SAP database growth control: data archiving general setup

Questions that will be answered in this blog are:

  • Which generic settings do I need to make for data archiving in the technology domain?
  • Why should I use a content server to store archive files?

Data archiving content server setup

For data archiving you can use the file system for storing the archive files. This you can do to perform initial testing. For productive use it is best to store the archive files in a content server. It will not be the first time an overzealous basis person in need for file storage deletes some old files in a directory called /archive…..

After you install the content server, set up in OAC0 the customizing for the content server to use it for Archivelink:

OAC0 define content server

Data archiving general technical settings

Now start transaction SARA:

SARA start screen

In this initial screen no object is selected. Now press the Customizing button.

Data archiving customizing

Set the Cross-Client File Names/Paths to your needs. You can do that from this menu, or directly from the FILE transaction.

Set the physical path name to be used:

ARCHIVE_GLOBAL_PATH FILE name

Even when you use content server the file will first be written to physical path for temporary storage.

And check the archive file name:

ARCHIVE FILE name

Technical settings per archiving object

Per archiving object you can set the technical settings. Normally you keep settings the same per object. Only for very large installations with archiving or special needs, you might want to deviate.

In the technical settings per data archiving object set the following:

Data archiving technical customizing per object

Important settings to set:

  • Max size in MB or the max objects
  • Check the variants (some variants for production have still deliberately the test tick box as on: you have to change it)
  • Best to leave the delete jobs to Not scheduled (large archiving runs can create many files and many deletion jobs to kick in at the same time): best to do this manually in controlled way
  • Start storage automatically or manually is a choice for you
  • Best to store before deletion. This is the most conservative setting.
  • Best to delete only from storage system: if file is not stored properly in any way, deletion will not have. This is the most conservative setting.

2018 improvement notes on Data Archiving

In 2018 SAP released several improvement OSS notes on data archiving. Description can be found in this blog.