The Datarock Core platform as a customer API that can be used to transfer between Datarock Core and our customer’s databases. This article describes the export functionality and provides a typical workflow that can be used.
Asynchronous exporting overview
Asynchronous exporting is a widely used design pattern in systems where generating export files—such as CSV, JSON, or PDF—may be computationally intensive or time-consuming, often due to the volume of data or the complexity of processing involved. In such cases, completing the export within a single synchronous API request could lead to timeouts, poor user experience, or undue strain on system resources.
To address this, asynchronous exporting separates the initiation of the export process from the retrieval of the finished export file. When a client requests an export, the system immediately acknowledges the request and begins processing the export task in the background. This decoupling allows the API to remain responsive and scalable, as it does not block client interactions while the export is being generated. Clients can then check back periodically to determine whether the export is ready for download. This model is particularly effective for handling large datasets, supporting concurrent requests, and maintaining system stability under load.
Suggested initial flow
This section provides a suggested initial flow for customers aiming to export data from Datarock Core for the first time. The final flow may vary depending on the specific use case.
Referenced endpoints are detailed in the Datarock Core API Documentation.
1. Export a list of projects
Use the GET /projects endpoint to find your project UUID(s). Most customers only need to do this once at the very start of their integration, because the UUIDs never change.
2. Export a list of Hole IDs
GET /projects/{projectUuid}/holes returns a list of Hole IDs associated with the project. You can use these to create export requests in our system for specific holes. The Hole IDs never change, but obviously new holes can be added to the project.
3. Request export to be created
Ask our system to create an export for you using one of the request endpoints.
If you want RQD, the request endpoint is POST /export/artefacts/rqd. This is the endpoint that actually creates an export for you. This will return an exportId you can use to fetch the exported artefacts, once the export is finished.
You can also pass filters to this request to tell the system which holes you want to download, for which dates. For example, you can ask for RQD artefacts that were generated yesterday from holes DDH021 and BHUG035. Nothing else will be returned in the export. If you want to download artefacts for every hole in your project, then you'll need to pass all the Hole IDs to the request endpoint. See the documentation for exact parameters.
4. Export artefacts
Because the exports can be quite large, our system needs some time to prepare them (usually 10s of seconds, but if you request artefacts for every hole, it may take minutes, for example).
If you want, you can use the GET /export/artefacts to track the status of the export.
Once complete, you can use GET /export/artefacts/{exportId} to download the artefacts.