Why this is the default architecture
- API keys stay private
- retries and polling stay in one place
- run IDs can be stored durably
- your application can normalize results before exposing them to users
Recommended flow
- your application sends a request to your backend
- your backend calls Bulkgrid
- your backend stores the run ID for async workflows
- a worker or scheduled task polls for completion when needed
- your backend returns normalized results to the rest of your system
Example
Async workflow recommendation
For extraction, crawl, and deep crawl:- create the run in an API route or job handler
- persist the run ID and customer context
- move polling to background work when the user should not wait inline
- return a clean status model to your own frontend