AI Data Architecture for Commodities Firms
I speak to my AI agents constantly, so quite a bit of my personal information gets passed to OpenAI servers. Personally I don’t really care if OpenAI knows I got bloated from coconut shrimp soup, so that’s fine.
Commodities firms, however, could be feeding AI information that is far more valuable and sensitive: live positions, PnL, inventories, contracts, credit limits, etc. Some of that data are arguable so critical that they should never leave the company.
A tension naturally arises for a firm where:
- You need to give your model enough data for it to do a task well.
- But the more data you feed into a model, the greater the risk of that data being exposed, mishandled, or leaked.
i.e. You want to run AI, but you also want to control where your data goes.
This is where data architecture comes in. There is a spectrum of possible AI deployment options to choose from, in terms of how tight you want the data boundary to be:
If you have no time to digest the whole table, the crux to understand is basically:
- Enterprise AI SaaS = outsource the application + inference.
- Managed AI service = control the application and surrounding data architecture, but outsource inference.
- Self-managed model = control the application + inference.
Essentially, whenever you need to use a closed-weight model (i.e. ChatGPT/ Claude/ Gemini) you need to send your data to the provider for them to run inference.
When I think about commodities firms, be it trading houses or hedge funds, I feel that a lot of the internal data is sensitive enough that sending it to someone else’s servers feels uncomfortable, even if enterprise AI SaaS providers promise strict limits on how that data is handled.
On a side note, this makes legal negotiations around provisions covering data training/ retention/ human access in enterprise AI contracts extremely critical in a closed source world.
There is an alternative though, that does not require firms to trust external providers altogether.
A Future of Local Models: Open Source + Adjusted Weights becomes your edge
In my imagined ideal world, the end state would be for firms to run their own models on their own infrastructure. That would allow all data to remain within the firm’s environment, making it much easier to give models access to all the data they need to perform well.
This can be accomplished most easily by using open sourced models as a starting point, and then fine-tuning these models on your firm data. The adjusted weight models then become your firm’s proprietary edge.
This future might not be so far away as open weight models continue to improve.
Scroll horizontally to compare every field →
| Deployment | Data flow | What | Operated by | Can the AI provider use your prompts, files or outputs to improve its models? This is separate from your firm deliberately fine-tuning a model. | Data boundary | How much must your firm rely on an outside company to protect the data and operate the system correctly? | Effort | Cost | Models | Best for |
|---|---|---|---|---|---|---|---|---|---|---|
| 01Public AIConsumer tools and public APIs | ⇄⇄ | ChatGPT, Claude, Gemini (SaaS / API) | Vendor | Depends on product, plan and settings | Data leaves the firm | Highest | Very low | Subscription or usage | Latest proprietary + some open models | Public data, individual use and proofs of concept |
| 02Enterprise AI SaaSVendor-hosted workspace with enterprise controls | ⇄⇄ | ChatGPT Enterprise, Claude Enterprise, Harvey or hosted Mistral | Vendor | No, under enterprise terms | Data enters a vendor-managed environment | High | Low | Enterprise contract + usage | Vendor models or vendor-selected underlying models | Fast rollout with enterprise access and audit controls |
| 03aPrivate cloud: managedPrivate network path to a managed model service | ⇄⇄⇄ | Azure OpenAI or AWS Bedrock | Cloud / model provider | No, under enterprise service terms | No public-internet route; provider still runs inference | Medium — provider code and control plane | Medium | Usage + managed-service fees | Provider catalog: proprietary and open models | Regulated firms that need controls without running GPUs |
| 03bPrivate cloud: self-managedYour serving stack on GPUs in your cloud account | ⇄⇄ | vLLM or similar on cloud GPU instances | Your firm | No provider receives the data | Data stays in the firm-controlled cloud workload | Lower — cloud infrastructure remains external | Medium–high | GPU capacity + operations | Open-weight models such as Llama, Qwen or Mistral; your firm can fine-tune the weights | Data residency, model choice and serving control |
| 04On-prem / isolatedA firm-run model on local infrastructure | ⇄⇄ | Open-weight model on firm-controlled GPUs | Your firm | No provider receives the data | No external transfer by design | Lowest | High | Hardware + full operations | Open-weight models; your firm can fine-tune the weights | Highly sensitive, restricted or sovereign data |
Products can span more than one row. Harvey is enterprise AI SaaS. Mistral can be public SaaS, enterprise SaaS, private-cloud or self-hosted, depending on the deployment selected.
Private endpoint ≠ self-hosted. It provides a private route to a provider-managed service; it does not move the provider’s inference stack into your VPC.
Hosting ≠ training. A local or open-weight model keeps its existing weights unless you deliberately run fine-tuning or another training process.