Whenever you run an Apify actor, the system automatically creates an unnamed key-value store that can be used to get and set data related to the run simply by calling the Apify.getValue
and Apify.setValue
functions provided by the apify NPM package. However, sometimes you might want to store data to or fetch data from a named key-value store that is shared between multiple runs of the actor or between multiple actors. In this tutorial, you'll learn how to do that.
The Apify.client
property contains a reference to an instance of the ApifyClient
class provided by the apify-client NPM package.
First, let's call the getOrCreateStore
function of the ApifyClient.keyValueStores
object to obtain the ID of a key-value store specified by storeName
:
Apify.main(async () => {
const { keyValueStores } = Apify.client;
// Destructure 'id' and name it 'storeId'.
const { id: storeId } = await keyValueStores.getOrCreateStore({
storeName: 'My-named-store'
});
// Write your code here...
});
Next, make the storeId
the default store used by the client, in order to avoid the need to pass it to every function called on keyValueStores
:
// Set current 'storeId' as the default used by the client
Apify.client.setOptions({ storeId });
Now, every method we call on the keyValueStores
will affect the selected named store. For example, you can get a record by calling:
const record = await keyValueStores.getRecord({ key: 'My-key' });
Example:
In this example, let's say we have an actor to send emails whenever necessary, e.g. if website content has changed. We want to avoid repeated sending of the same emails, so we store the state of the sending to a key-value store named 'Emails' in a record with the key 'STATE':
Apify.main(async () => {
const { keyValueStores } = Apify.client;
const { id: storeId } = await keyValueStores.getOrCreateStore({
storeName: 'Emails'
});
Apify.client.setOptions({ storeId });
// Just pass the 'key' as the 'keyValueStores' already knows
// in what 'storeId' to look at.
const record = await keyValueStores.getRecord({ key: 'STATE' });
// Check for empty 'STATE' records
const storeRecord = record && record.body ? record.body : {};
const previousState = typeof storeRecord === 'string' ?
JSON.parse(storeRecord) : storeRecord;
});
We do our magic in between: check for repeated records, remove unwanted ones, format...
And finally we set a nextState
as the next record in our store.
Apify.main(async () => {
const { keyValueStores } = Apify.client;
const { id: storeId } = await keyValueStores.getOrCreateStore({
storeName: 'Emails'
});
Apify.client.setOptions({ storeId });
// Just pass the 'key' as the 'keyValueStores' already knows
// in what 'storeId' to look at.
const record = await keyValueStores.getRecord({ key: 'STATE' });
// Check for empty 'STATE' records
const storeRecord = record && record.body ? record.body : {};
const previousState = typeof storeRecord === 'string' ?
JSON.parse(storeRecord) : storeRecord;
// Magic here...
// It is a good practice to copy objects instead of
// overwriting them. Weird things can happen otherwise.
const nextState = Object.assign({}, previousState, stateChanges);
await keyValueStores.putRecord({
key: 'STATE',
body: JSON.stringify(nextState),
});
});
And that's it. Here's a quick recap:
- getOrCreateStore
- setOptions
- getRecord / putRecord / deleteRecord / listKeys