I have a fairly straight forward question about the PX API .Gainsight PX REST API · Apiary
When trying to execute the Get Custom Events call, the returned scrollId has a longer set of characters than Postman will allow.
Does anyone have any recommendations on how to workaround this? Additionally, I am a novice here, so wondering what I can do to continually loop the returned scrollId into my next call. The goal is to return all 4 million records into a single JSON file.
+4
Hi
Welcome to the group & thank you for dropping in your question!
I’ve turned your interesting question on the previous thread into a new post so more members of the group can see it and provide their thoughts as well. 
(Works best to always post your questions/tips as a new topic in this 'Calling API Callers' Group, instead of a comment on a thread, so that people are more likely to see it).
+3
Hi JP,
I am definitely not an expert with the PX API specifically, so hopefully others will respond to your question. But, I am attempting to understand your use case. APIs are generally very good at extracting or updating small amounts of data. If you’d like to extract a large amount of information, have you considered performing a Data Export directly within PX?
Hi
PX has limitations on how much global context you can see within the querybuilder view plus limitations on download. We are advised to use the APIs to create a larger dataset to generate deeper datamodels of the PX data.
I am waiting for someone to recommend a method for
cc:
Here is an example of conflicting messages:
My team is currently taking advantage of the standard S3 export integration to create tables dedicated to this purpose.
But for larger analysis of details in custom events, it is best to extract the dataset on an as needed basis. Postman is a layman’s (non-coder) path for quick retrieval of large amounts of data from GPX. But on occasion, as with this request, I am looking to you coders out there to help me figure out how to accommodate for the lengthy page ids and loop them back into the API call so I can get a few hundred thousand records into one JSON file.
+2
Hi All!
Gainsight PX data exports are available to send to the configured AWS S3 bucket in CSV, JSON, and XML format through our PX Amazon S3 export integration and includes full listings for PX Accounts, PX Users, PX Features, and Engagements as well as daily rollup aggregations for Feature matches, Custom Events, Pageviews, Engagement views, Survey Responses, and Feedback events, etc. Everyone can review the linked online documentation above for more specific details about what is currently available.
Gainsight PX Enterprise level customers as well as our CS+PX better together customers have access to our advanced Bulk Export Custom Reports feature that allows them to build a limited number of additional custom AWS S3 exports (CSV format) of daily rollup aggregations of specific Custom Events, Product Map Modules/Feature matches, Engagement views, Session Initialization, and Page View events filtered and/or grouped in advanced ways using different PX Account/User attributes, Global Contexts, Inferred location, and device types. Everyone can review the linked online documentation above for more specific details about what is currently available.
With both PX in app data export features above, once they are configured, PX will automatically begin to send those export files to the chosen AWS S3 buckets on a nightly basis going forward and they will include the previous day’s rollup aggregations. If you initially need a larger, custom set of date ranges included (i.e the last three months), you can submit this one-off custom export request to our amazing PX Support team for more assistance.
What makes the PX REST API fundamentally different than our Gainsight PX data exports is that one can actually use it to retrieve the individual raw events captured by PX for items like Feature matches, Custom Events, Identify events, Segment matches, Sessions initialization, etc. and not daily rollups/aggregations. One can also perform the necessary CRUD (Create, Read, Update, Delete) operations with this too. Everyone can review the linked online documentation above for more specific details about what is currently available.
Everyone also needs to be aware the our PX REST APIs have limits and can not / should not be abused. Thanks in advance!
I have included all of this above, just so I can help as much as possible with everyone’s general PX REST API questions AND to try to answer
I have never heard of a scrollId limit specifically in Postman, but it does seem that each Postman URL does have a limited length … cannot exceed 1024 characters if I am only trusting The Google here.
In my testing using our PX REST API’s Get Custom Events method it appears that the length of the returned scrollId is consistently 848 characters long. So, if that Postman URL length limit is correct, then that really does not leave much room to add any additional filters for items like eventName, identifyId, date, sort, or scrollSize parameters. See sample below:
I am not sure if you can confirm, but if there is a way for you to only use the pageSize and scrollId URL parameters and not (many) other filters, you might be able to continue using PostMan for this extract. See below.
Postman does offer Scripting capabilities and there are other scripting tools available for intermediate developers to leverage like Python, which we use for our PX Data Loader, to make these extracts less human-effort required. Hopefully everyone is exploring these options for these larger PX REST API data extracts.
I hope all this helps some, but do comment here with additional details.
Happy PX-ing!!!
And you are spot on! It must be the URL length that I am getting snagged on. I reduced the number of parameters and it worked just fine.
Cheers!
+1
OMG
Thank you!
Scott Drost
@link
Hi Link, I noticed that the Work with the Gainsight PX REST API page will be unpublished shortly. Will we be directed to use the https://gainsightpx.docs.apiary.io/# site instead?
+2
?
+5
Saving this thread for future reference! Thank you!!
If you ever had a profile with us, there's no need to create another one.
Don't worry if your email address has since changed, or you can't remember your login, just let us know at community@gainsight.com and we'll help you get started from where you left.
Else, please continue with the registration below.
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
