Straighforward Data Snapshots

If your app has a lot of content, chances are that by the time you get a chance to work on a bug report, the data that the bug appeared on will be long gone.

Here are some tidbits on how I created a simple solution for that problem at The New York Times, it’s based on a simple idea.

Our content changes very often and since we work in agile methodology, by the time we get a bug brought to next sprint the data that caused it is usually gone from the main screen on our app (Section Front).

In the past it forced our team to use staging environment and try to bring the article that caused bug back to be featured, but our teams have grown quite a lot and getting articles featured at right spots is very problematic and time-consuming, not to mention that it required a lot of manual steps for each bug report and one person could override article that another one needed.

Last year I’ve decided that I’ve had enough of that, and instead of relying on our servers I decided to automatically record currently viewed data and upload it to remote servers whenever a bug report was created.

Snapshot Mode

Regardless of what your app does, almost all apps have a point where they deal with raw Data object, and you should be able to give any Data operation a unique identifier.

I wanted a simple tool that can be dropped-in into any project, at Times or anywhere else, without requiring any significant architecture changes, a simple and pragmatic solution.

Why not just pipe that data through an object that can decide what to do with it?

public class SnapshotMode {
  public enum State: String {
      case disabled
      case recording
      case replaying
  }
  func process(data: Data, uniqueIdentifier: String) -> Data
}

the process function is quite straightforward:

• recording mode: it saves the Data to a selected folder on disk, and returns input data
• replaying mode: it ignores passed in Data and returns the data from the folder on disk
• disabled: returns input Data

Organizing data

Since you can have a lot of operations, and not all of them might fire at the same time, I wanted to make sure I know which ones are available at any point, I did it by simply introducing module registry to SnapshotMode:

snapshotMode?.register(module: "Home", files: [
  "UpdateParsedHomeDataOperation-CacheableHome",
  "UpdateFeedPresentationConfigOperation"
])

This module-info is evaluated against recorded operations and can be viewed in our beta settings.

Usage

We use a lot of operation queues, and we use an idea of Current environment.

So to adapt to this snapshot idea, the only thing I needed to do is add the following lines in each operation I wanted to allow being snapshotted:

let data = Current.snapshotMode.process(
  data: try JSONSerialization.data(withJSONObject: responseObject, options: []),
  uniqueIdentifier: "\(String(describing: type(of: self)))-\(operationName)"
)

Beta Settings

We automatically upload snapshots with each bug report, but you can also manually do that via our Beta Settings.

You also use Beta Settings to download and activate a snapshot from bug report by simply entering its UUID.

Uploading / Downloading

You can simply zip your selected folder and upload it to your server instance, and perform the reverse when loading a specific snapshot.

Conclusion

This simple idea has saved our developers a lot of time and didn’t require architectural changes in our project so it was easy to adapt and it’s simple to remove if we ever want to.

I’m not open-sourcing our full implementation because I can’t maintain another project but if I see there is an interest in this kind of writing I’m planning to share some other ideas I’ve built over last few years.