Memory Database Save Data v7

• status: complete
• version: 7.x
• follows from: Memory Database Select Queries

Saving the Memory Database to a File

You can save the content of the memory database with the save and saveSync commands:

// Store reference.
let memory = node.game.memory;

// Asynchronously save all items.
memory.save('data.json');

// Synchronously save all items.
memory.saveSync('data.json');

You can also save a subset of the items in the database.

// Save data from stage 1.
memory.stage['1.1.1'].save('data_stage1.json');

// Save data from player with id 'A'.
memory.player.A.save('data_player_A.json');

// Save the result of a custom query.
node.game.memory
    .select('value', '>', 0)
    .or('value', '<', 5)
    .save('data_values_05.json');

By default all the data is saved under data/roomXXX/, where XXX is the room number.

CSV Files

If you prefer to save to a CSV (comma separated value) file, you can simply give the extension .csv.

memory.save('data.csv');

If your data contain objects, their first-level properties are automatically expanded and written under a new column named topPropertyName.nestedPropertyName.

For instance, assuming you have just one object in the memory database:

// Manually add an item containing objects.
memory.add({
    player: 'A',
    stage: { stage: 1, step: 1, round: 1 },
    data: "A",
    nestedData: { decision: "GO", time: 1234 }
});

and you save it to a csv file:

memory.save('data.csv');

you will get:

"player","stage","step","round","data","nestedData.decision","nestedData.time","timestamp"
"A",1,1,1,"A","GO",1234,1587540763230

Note that the "stage" object was decomposed into columns "stage", "step", and "round" and not "stage.stage", "stage.step", and "stage.round" (like for property nestedData). This is because a default adapter for the "stage" property is already preset to handle its nested values.

Specifying an Adapter for Items Containing Nested Properties

If your database contains objects nested at levels deeper than two, such in the example below:

// Adding an item with an object nested in an object.
memory.add({
    player: 'A',
    stage: { stage: 1, step: 1, round: 1 },
    data: "A",
    nestedData: { decision: "GO", time: 1234 },
    doubleNestedData: {
        level1var: "level1",
        level1: {
            level2var: "level2",
            level2: {
                level3: "Hello World"
            }
        },
    }
});

and you save it to a csv file:

memory.save('data.csv');

you will get an [Object Object] reference:

"player","stage","step","round","data","nestedData.decision","nestedData.time","doubleNestedData.level1var","doubleNestedData.level1","timestamp"
"A",1,1,1,"A","GO",1234,"level1","[Object Object]",1587540763230

That is, the object nested inside doubleNestedData.level1 is not automatically expanded for performance reasons. You will need to specify an adapter as option to the save method.

The adapter is an object whose properties are functions returning a value to save under a column named after the property name. For instance:

// Saving the database with an adapter;
memory.save('data.csv', {
    adapter: {
        'doubleNestedData.level1': function(item) {
            // Return the value of interest from the double nested structure.
            return item.doubleNestedData.level1.level2.level3;
        }
    }
});

will generate:

"player","stage","step","round","data","nestedData.decision","nestedData.time","doubleNestedData.level1var","doubleNestedData.level1","timestamp"
"A",1,1,1,"A","GO",1234,"level1","Hello World",1587540763230

Notice that adapter function extracted only one variable from the double nested object. If you wish to extract more values, simply add a new property to the adapter.

Usually, only some specific items in the database will have a deep nested structure such as in the example above. So, you need to make sure that the adapter function can handle any item in the database.

// Adding another object without the double nested structure.
memory.add({
    player: 'A',
    stage: { stage: 2, step: 1, round: 1 },
    data: 'B'
})

// Improving the adapter to handle any item.
memory.save('data.csv', {
    adapter: {
        'doubleNestedData.level1': function(item) {
            // In case the item does not contain a double nested structure.
            if (!item.doubleNestedData) return 'NA';
            // Return the value of interest from the double nested structure.
            return item.doubleNestedData.level1.level2.level3;
        }
    }
});

And you will get:

"player","stage","step","round","data","nestedData.decision","nestedData.time","doubleNestedData.level1var","doubleNestedData.level1","timestamp"
"A",1,1,1,"A","GO",1234,"level1","Hello World",1587540763230
"A",2,1,1,"B","NA","NA","NA","NA",1587540763231

Specifying a Custom Header

You can save only a subset of the properties contained in the items in the database by specifying a custom header.

In the example below all the properties not specified in the header will be ignored.

memory.ultimatum.save('ultimatum.csv', {
    // Specify header in advance.
    header: [
        "player", "stage.round", "role", "partner", "offer", "response"
    ]
});

Keeping CSV Files Updated

To automatically save to the file system every new entry added to the database (as well as views and hashes) use the keepUpdated flag.

Let's assume that participants leaves their feedback at the end of the experiment, but this happens at random intervals. Here is the code snippet to automatically save them:

// Create a new 'feedback' view and save all updates to CSV file.
db.view('feedback').save('feedbacks.csv', {

    // Specify a custom header.
    header: [ 'timestamp', 'user', 'feedback' ],

    // Incrementally save to the same csv file all new entries.
    keepUpdated: true,  

    // As new items are generally clustered in time, it is good to add some
    // delay before saving the updates. Default: 10000 milliseconds
    updateDelay: 5000    
});

Alternatively, if you know already when a new set of feedbacks are added to the database (e.g., at the end of a round for all participants) you can manually control when to save the new updates, using the updatesOnly flag.

// Feedback view already created.
db.feedback.save('feedback.csv', {

    // Custom header.
    header: [ 'timestamp', 'user', 'feedback' ],

    // Saves only updates from previous save command.
    updatesOnly: true
});

Flatten Items to Save in CSV

To save all the items of a stage with multiple steps and multiple players with one row per participant in the CSV file, you can use the flatten flag.

stager.extendStage('ultimatum', {
    init: function() {
        // Creates a view for items in the ultimatum stage (all steps).
        // We will use it to save items after each ultimatum round.
        memory.view('ultimatum', function() {
            return node.game.isStage('ultimatum');
        });
    }
});

stager.extendStep('ultimatum', {
    // The `exit` callback is executed after a step is finished.
    exit: function() {
        // Assuming a view
        memory.ultimatum.save('ultimatum.csv', {

            // Specify header in advance.
            header: [
                "session", "player", "stage.round",
                "role", "partner", "offer", "response"
            ],

            // Merge items together.
            flatten: true,

            // One row per player.
            flattenByGroup: 'player',

            // Adds updates to same file for every round.
            updatesOnly: true         
        });
    }    
});

Next

• Learn about the Game Setup file

Sources

• GameDB
• NDDB