RavenPack unveils AI-powered dataset pipeline to test investment ideas

RavenPack has launched a new dataset-generation pipeline on Bigdata.com, its AI grounding and search platform, that can transform a research hypothesis expressed in plain language into a labelled, backtest-ready investment dataset.

Powered by a fine-tuned NVIDIA Nemotron model, the system is designed to help investment firms test ideas across vast volumes of financial content rather than relying on smaller samples. The company says this makes large-scale text labelling more practical while generating signals that go beyond traditional sentiment analysis.

The first application explored a simple question: whether a company is gaining or losing market share against competitors. To test the idea, the pipeline retrieved 845,220 excerpts from news reports and earnings-call transcripts covering 1,642 companies in a point-in-time Top US 1000 universe by market capitalisation. The content was labelled using a purpose-built taxonomy and converted into a daily company-level signal, which was then backtested from January 2016 to October 2026.

The process combines targeted retrieval and model distillation. Bigdata.com's search API uses smart batching technology to pull relevant information from an archive of more than one billion documents spanning more than 20 years of news and earnings-call transcripts, reducing query volumes by more than 90 per cent compared with a full company-by-period search approach.

For the labelling stage, RavenPack used NVIDIA Nemotron 3 Super 120B to classify a stratified sample of 84,521 excerpts before using those results to fine-tune NVIDIA Nemotron 3 Nano 4B through LoRA supervised fine-tuning. Running on two NVIDIA H200 NVL GPUs, the smaller model was then used to label the full corpus at both chunk and sentence level, achieving around 39 times the throughput of the larger model in RavenPack testing.

According to the company, the system labelled 845,220 passages in around 49 minutes. It also reduced the cost of labelling a full historical dataset by 63 per cent compared with a proprietary model delivering the same performance, including the one-off cost of distillation, while subsequent runs cost 96 per cent less.

On held-out data, the 4B model matched the teacher model on 94.05 per cent of chunk-level labels and 95.90 per cent of sentence-level labels. RavenPack also reported that the resulting sentence-level signal produced an information ratio of 1.18 and an annualised excess return of 8.63 per cent, compared with 0.41 and 3.78 per cent respectively for a relevance-weighted sentiment-only approach using the same portfolio construction framework.

The company said open-weight models were central to the project because distillation requires direct access to model weights for fine-tuning and deployment. It added that NVIDIA Nemotron's open-weight architecture enables organisations to fine-tune once and run repeated inference workloads, while also allowing regulated institutions to deploy and audit the model internally.

Peter Hafez, chief data scientist at RavenPack, said: "Data science teams are not short of ideas. The bottleneck has always been turning those ideas into reliable datasets with sufficient coverage and history. This pipeline changes that: clients can move from a hypothesis to testing it across the full breadth of content indexed by Bigdata.com.”

The pipeline is available immediately to Bigdata.com clients, with developers able to run the full workflow from research idea to labelled dataset, signal generation and backtest through a companion notebook.