API request template issues

theodolite

I'm looking to export all reports in my Expensify account in pdf format (there are thousands of them). The only way I can think to do that is by setting a date range in the inputSettings filters field and output the returnRandomFileName result to a CSV file. I'll then pass the CSV results into a separate script for downloading.

Script 1 should output the random file names for pass to a download script.

My template is just the default FreeMarker template.

I am able to run the curl request for Script 1, but it returns one of two errors in the CSV file each time:

"responseMessage":"No Template Submitted","responseCode":410}

"responseMessage":"Too many reports to be processed at once, template needs to use the \"addHeader\" statement.","responseCode":500}

These responses depend on how I format the data-urlencode field in Script 1. I've tried both template@expensify_export.ftl and template=expensify_export.ftl

It's as though the script is not passing the data from my template into the POST request, despite them living in the same working directory. Also worth noting this script works as expected when the filters are changed to reportIDlist instead of the date range.

Here is the code for Script 1 and my template. Any ideas on why Script 1 continues to error?

Script 1

#!/bin/bash


curl -X POST 'https://integrations.expensify.com/Integration-Server/ExpensifyIntegrations' \
    -d 'requestJobDescription={
        "type":"file",
        "credentials":{
            "partnerUserID":"myIDhere",
            "partnerUserSecret":"mySecrethere"
        },
        "onReceive":{
            "immediateResponse":["returnRandomFileName"]
        },
        "inputSettings":{
            "type":"combinedReportData",
            "filters":{
                "startDate":"2019-02-01",
                "endDate":"2019-03-01"
            }
        },
        "outputSettings":{
            "fileExtension":"pdf",
            "includeFullPageReceiptsPdf":"true"
        }
    }' \
    --data-urlencode "template=expensify_export.ftl" >> filenames.csv

My template

<#if addHeader == true>
    Merchant,Original Amount,Category,Report number,Expense number<#lt>
</#if>
<#assign reportNumber = 1>
<#assign expenseNumber = 1>
<#list reports as report>
    <#list report.transactionList as expense>
        ${expense.merchant},<#t>
        <#-- note: expense.amount prints the original amount only -->
        ${expense.amount},<#t>
        ${expense.category},<#t>
        ${reportNumber},<#t>
        ${expenseNumber}<#lt>
        <#assign expenseNumber = expenseNumber + 1>
    </#list>
    <#assign reportNumber = reportNumber + 1>
</#list>

Francois Laithier

Hi @theodolite,

To tell curl to load the content of your template file, you'll need to use the name@filename syntax, so in this case:

--data-urlencode 'template@expensify_export.ftl'

Cheers

Jenna Hay

Hi @theodolite

Could you please also past here the script which does work and export the PDFs as expected so we can investigate further?

Thanks

theodolite

Hi @Jenna Hay,

Sure, here is the script that does work with the reportIDlist referencing a single test report in my Expensify account. reportIDlist also works as a comma separated JSON array here. It uses the same template as above.

#!/bin/bash


curl -X POST 'https://integrations.expensify.com/Integration-Server/ExpensifyIntegrations' \
    -d 'requestJobDescription={
        "type":"file",
        "credentials":{
            "partnerUserID":"IdHere",
            "partnerUserSecret":"SecretHere"
        },
        "onReceive":{
            "immediateResponse":["returnRandomFileName"]
        },
        "inputSettings":{
            "type":"combinedReportData",
            "filters":{
                "reportIDList":"58466904"
            }
        },
        "outputSettings":{
            "fileExtension":"pdf",
            "includeFullPageReceiptsPdf":"true"
        }
    }' \
    --data-urlencode 'template=expensify_export.ftl'

Here is the downloader script.

#!/bin/bash


curl -X POST 'https://integrations.expensify.com/Integration-Server/ExpensifyIntegrations' \
    -d 'requestJobDescription={
        "type":"download",
        "credentials":{
            "partnerUserID":"IDhere",
            "partnerUserSecret":"secretHere"
        },
        "fileName":"<replace with random generated file name from script 1>",
        "fileSystem":"integrationServer"
    }' >> test_report.pdf

Thanks for the help.

theodolite

Hi @Francois Laithier,

Thank you for the input. When I use the @ syntax instead of the = in the --data-urlencode field, this is the error I receive:

"responseMessage":"Too many reports to be processed at once, template needs to use the \"addHeader\" statement.","responseCode":500}

Francois Laithier

Got it. I see a few requests where the template seems to be passed successfully with the @ syntax, but that failed to generate all the PDFs. As a test, can you try to use a narrower time frame with the startDate and endDate params, for example 1 week?

theodolite

@Francois Laithier I narrowed the date range to just one day (just 44 report entries). I got an output of 44 file names as expected after 13 minutes and 41 seconds. Perhaps the issue is simply the volume of reports. If it takes ~15 minutes to do one day's worth, 1+ month could be many hours. Thanks for the suggestions. Seems the scripts/template is working as expected.