Improve Offline Capability and Reduce Data Sync Time

Use of mobile devices for plant or warehouse management has helped organizations digitize their data and improve efficiency. 

As we rolled out our platform to medium and large enterprise customers, we saw a strong use case for storing large data sets on the mobile for offline use. To address the very need and to improve the end user experience of the mobile user, we at Innovapptive focussed on capabilities that improved offline data capture and sync. Traditional methodologies of syncing full data sets into mobile do not scale well for these use cases, prompting us to build a solution to only move changed dataset into the product.

Observations

When we designed offline capability, the following parameters were considered:

• Performance: Datasets of gigabytes per person per device takes tens of minutes to pull to the device.
• Scale: High volume adds exponential load on the databases and slows them down.
• Data Integrity: Initial sync runs into tens of minutes during which time updates can happen to the data.
• Recoverability: Recovery from a network outage or device restart is critical to save time and ensure data credibility.
• SAP Platform Limitations:  Limited offline capabilities in both SMP/SCP and in the NetWeaver Gateway, which were not designed to synchronize large amount of offline data.

The Model

For one customer, a synchronization of 2.5 GB of Data took over 18 minutes with excellent bandwidth scenarios. Any improvement in efficiency required a scalable model that integrates with EAM and PM modules of SAP and: 

• Stored data offline so that you can make transactions when not in network.
• Moved only the changed datasets when the device was back in network.

Using this use case of moving only the changed datasets, the solution synchronized data in under 2 minutes (improvement of over 90-95% at BAU loads).

The Solution

The solution was designed based on the following significant aspects: 

• Multilevel expansion of objects that resulted in a n-tier object hierarchy.
• Incremental (Delta) changes between refreshes from the end user usage.

Model for Initial and Subsequent Refreshes

The solution follows the process below for initial synchronization and subsequent data refreshes. 

During initial Data sync (FULL SYNC)

Create a delta token and store it with a timestamp and device ID for each user per collection.  

1. Return a delta token to device for subsequent requests.
2. Run a delta sync to check for records that were created/updated during the initial sync.
3. Send updated delta token to the mobile device.

During subsequent refreshes (DELTA SYNC)

1. Fetch timestamp for the delta token produced by the device.
2. Fetch changes by running the query in the change pointer table.
3. Create an updated delta token for future requests from the device.  
4. Return the delta token to the device with the response. 

Highlights of the Solution

• Manages delta tokens on a per user and per device. 
• Provides for multiuser or cross user strategies across device types.
• Uses Defining Request paradigm of SAP Cloud Platform (SCP) to define delta retry strategy. The platform allows you to keep pulling delta data until there are no new changes.
• Uses SCP and SAP NetWeaver’s inbuilt mechanisms to generate ODATA requests with appropriate expand and delta tokens.
• Evaluates indices for change pointer table and carefully analyze and add indices. Depending on the dataset, the source of changed data might differ. This is especially true with delete scenarios. Provides for that change.
• Carefully analyzes the data that undergoes regular changes. For example, as Master data does not change frequently there is no need for it be regularly refreshed.
• Uses UTC for time.

When implemented right, delta sync can be a huge time saver for mobile users on the go. At Innovapptive, we are proud to provide yet another way to improve the labor productivity.

Contact us to understand how to reduce data synchronization timelines and increase productivity.