Not Planned

Export of >100000 records from Horizon Analytics reports

  • 25 August 2021
  • 7 replies
  • 32 views

Userlevel 6
Badge

Data download from Horizon Analytics reports is apparently limited to 100000 records (lines) and according to documentation, export to S3 via Rules Engine should be used when more records need to be exported. This makes it really hard for end-users without Rules Engine access to export large dataset (e.g., usage data) from reports to do further analysis.

The option to export to csv (via an email link when the file was ready) available on the old reports was great for these purposes. I'd be very happy to see this option (or another way to easily download larger datasets) return.


7 replies

Userlevel 7
Badge +2

@Jef Vanlaer I will get back to you with the limitation. Thanks.

Userlevel 6

@Jef Vanlaer Thanks for the input. A clarification on this - we never supported export of reports with more than 100,000 records. There are two reasons for this

  • When you are exporting that much amount of data, it is more a data export than a report export and 
  • Most importantly, processing that much data will negatively impact the user experience of other users who are exporting reports as they will have to wait longer than expected to download their reports.
Userlevel 6
Badge

@rahul_prayakarao I was under the impression that that second reason explained why larger exports were provided via email before. This way, they can be scheduled and processed in the back-end, so performance is not negatively impacted for other users.

Userlevel 6

@Jef Vanlaer If I understand your request correctly, it will be sufficient if we give this capability while only scheduling the report exports and not while downloading it directly?

Userlevel 6
Badge

@rahul_prayakarao Absolutely

Userlevel 6

Thank you for the confirmation. Please give me some time to validate this request and get back to you.

Data download from Horizon Analytics reports is apparently limited to 100000 records (lines) and according to documentation, export to S3 via Rules Engine should be used when more records need to be exported. This makes it really hard for end-users without Rules Engine access to export large dataset (e.g., usage data) from reports to do further analysis.

The option to export to csv (via an email link when the file was ready) available on the old reports was great for these purposes. I'd be very happy to see this option (or another way to easily download larger datasets) return.

We often suffer the same issue with our end users when exporting customer usage reports. Fully understand the performance impact but the proposed alternative with a link would be just fine.

Reply