Exporting Events using API

Answered

Follow

Steve Salt

• January 11, 2019 16:36

I'm looking at exporting the Events data (and other too, but focusing on Events):

Using the "Try it now" with a StartAt and EndedAt date range and a Limit of 50, I get data that starts with:

"response": { "count": 3697, "items": [ {

However, there are not 3697 results, there are 50. Which is what I expected.  If I remove the limit, I get 500 back, and Limit prevents me exceeding 1000.

So I thought I would try to chunk the data using Offset & Limit, but Export/Events.json doesn't use Offset.

Perhaps I could iterate based on the StaredtAt using a Date & Time, as the description says I can:

startedAt

Only include events that happened after this date (YYYY-MM-DD, YYYY-MM-DD HH:mm:ss or timestamp)

I get back:

"errorType": "general", "description": "Parameter startedAt is not valid. Ensure format is YYYY-MM-DD.",

 

With Export/shares.json, there are no parameters - I just get all the shares, 1272 so far and counting. I like that, its simple.

So, my question is 

What is the suggested way to export all the Events without loss?

 

Regards,

Steve Salt

Application Devloper

0

• Official comment

  

  Elaine Mui

  • January 14, 2019 18:23

  Hi Steve,

  The default maximum number of returned items in the Events API endpoint is set to 500. As you have found out, you can change the limit and set a range between 50 and 1000 responses.

  There are millions of events in the database and so we have not made it possible to return everything in a single request. There is a way to fetch this information in batches that you can scroll through until you have all the events that you need.

  The response comes with a header called X-Showpad-Scroll-Id that contains a token and remains valid for a limited time. It is comparable to an authentication key,  If you make the same request again but set this header on your request, you will get the second batch of events. You get back another X-Showpad-Scroll-Id that you can use to fetch the third batch of events etc. This way, you can do a loop in the application until it fetches all the events that you want.

  You can learn more about the X-Showpad-Scroll-Id header in our Help Center article. I hope this explains why you aren't able to get this information in a single export and gives you the information you need for your reporting.

  - Elaine

  Comment actions Permalink
• 

  Steve Salt

  • January 15, 2019 15:36

  Hi,

  I obtain the data using cURL as an Ajax request from within a PHP application. For anyone else doing this, you will need to set the following to get access to the header, and specifically, the X-ShowPad-Scroll-Id.

  curl_setopt($this->curl, CURLOPT_HEADERFUNCTION, array (&$this, 'methodName')); 

  Regards,

   

  Steve

  0

  Comment actions Permalink

Please sign in to leave a comment.