Export large amount of data issue as JSON

gg1234

I am trying to export a large amount of data by using a custom template so that the exported result is a JSON (since I prefer to have nested data instead of a flat csv). Given the big amount of data I got the following error:

"Too many reports to be processed at once, template needs to use the \"addHeader\" statement."

Do you have any idea about what to add if my template is JSON and not a CSV (as per API Documentation examples)?

The only workaround I can see is to call many times the api, but it can break at any moment.

Thanks in advance

gg1234

Maybe can be useful to other people: a solution is just to have smaller date ranges and iterated over them.

Sasha Kluger

Hi @gg1234 - For JSON export templates, you can use addHeader and addFooter for large exports. Here is an example of how this might look, but please note that this is just a rough example and isn't tested.

<#if addHeader>
[
</#if>
<#list reports as report>
  {
     details regarding each report
  }<#if report?has_next || !addFooter>,</#if>
<#if addFooter>
]
</#if>

gg1234

Thank you, I will try it

vishal22patel

@gg1234 @Sasha Kluger Did you able to pull a large amount of data using API without breaking the time period or is there any other way to pull more than 300 records? We are using Report Exporter to pull reports. The startdate has been passed into "inputSettings" filters (Startdate = 3 years from now). I have posted it to the community here.

John Schuster

Hey there @vishal22patel! Would you be willing to share the exact request you've written (minus the partner user secret and password)?

I'd be happy to review it!