AgFlow Forward Curves FAQ: Methodology Explained
Reading time: 5 minutes
Risk managers and traders use AgFlow Forward Curves for various reasons.
For risk managers, AgFlow Forward Curves are essential for checking and challenging traders’ positions through a series of processes, including:
- Checking the exposure
- P&L (Profit & Loss) reporting utilizing an MTM (Mark-to-Market) approach
- Determining the Value at Risk (VaR)
- Ensuring positions are correctly hedged
- And more
For traders, AgFlow Forward Curves are valuable tools to assess market movement and predict (or react) to market signals.
The fact is that most forward curves are incomplete due to the lack of data availability from brokers or the low update frequency of a given commodity. This is why AgFlow has decided to fill the gaps of missing forward prices by reconstructing complete forward curves with forward prices computed by our Machine Learning algorithm.
- How Does AgFlow Collect Data for Forward Curves?
- How Do AgFlow Forward Curves Differ from Regular Forward Curves?
- Why Does AgFlow Use the Interquartile Range (IQR) Method?
- Why Doesn’t AgFlow Use Linear Interpolation, Akima Interpolation, or ARIMA?
- How Does AgFlow Handle Seasonality and Market Changes?
- How Does AgFlow Backtest Forward Curves?
- What Are the Differences Between Regular Forward Curves and AgFlow Forward Curves?
- Which Tickers Do AgFlow Forward Curves Cover?
How Does AgFlow Collect Data for Forward Curves?
The data we use to reconstruct AgFlow Forward Curves follows a stream. We obtain cash quotes from trusted brokers or public sources, like Agropa Trading or USDA, in a variety of formats, from PDF reports to Excel spreadsheets. This data then goes through the parsing process, which automatically processes data into the database in the desired format. Once stored in our database, it is ready to be used for the AgFlow Forward Curves processing.
How Do AgFlow Forward Curves Differ from Regular Forward Curves?
Our AgFlow Forward Curves differ from what is found elsewhere on the market. Regular forward curves, based on forward contracts, depend on the trade volume of their commodity.
Typically, PRAs have improved these by relying on market analysts’ professional judgment, allowing them to fill some data gaps.
AgFlow goes even further. Our forward curves are an aggregation of cash and futures quotes that we complete with synthetic data.
First, the data is preprocessed using a simple IQR method to filter outliers. Then, quotes data are aggregated using time interpolation and Spearman correlated futures to have the most negligible error (or incertitude) possible. Once aggregated, the data passes through a Machine Learning algorithm to:
- Understand the dataset
- Create artificial data points predicted from training
- Test the datasets
- Back-test the curves every day to refit the data and improve the model
It then produces the best potential forward curves, which are perfected even further with the backtesting.
Original Incomplete Data for Argentina Feed Barley FOB Argentina Forward Curves
AgFlow goes even further. Our forward curves are an aggregation of cash and futures quotes that we complete with synthetic data.
Fully Reconstructed Data for Argentina Feed Barley FOB Argentina Forward Curves
Why Does AgFlow Use the Interquartile Range (IQR) Method?
Commodities data are subject to speculation and supply and demand like any trade in the world, and as such, there isn’t a unique pricing for a given commodity. This leads to a wide range of prices quoted every day for the most popular commodities. Moreover, price imputation, or currency and unit conversion mistakes, can sometimes lead to extremely low or high prices.
This variety of prices also leads to some uncertainty when producing forward curves. It is necessary to filter the data in the preprocessing to remove these outliers and have the least uncertainty possible to ensure that the curve is not biased by extreme prices that could lead risk managers or traders to draw wrong conclusions.
However, more complex statistical techniques can remove even more data, which produces the opposite effect of smoothing the curves too much, resulting in information loss and making market movements and signals difficult or impossible to view.
Additionally, this preprocessing takes time to produce. But forward curves need to be reprocessed daily, and the more refined the techniques, the more time this task takes. Thus, it was necessary to find a good balance between time consumption and uncertainty in curves. Testing both parameters led us to choose the IQR (Inter-Quartile-Range) method, as it was the best compromise.
Why Doesn’t AgFlow Use Linear Interpolation, Akima Interpolation, or ARIMA?
Before inputting the data into the Machine Learning model, we first aggregate cash and futures quotes. The reason behind this is that datasets are not complete. If incomplete, this can lead the model to be less performant and thus less accurate.
When aggregating the data, the dataset must follow the price curves in time. In commodities data, prices can change drastically due to seasonality or market structure changes. With this in mind, it is necessary to limit the Mean Absolute Error (MAE) value, which would artificially increase the uncertainty and accuracy of prices over time.
Therefore, similarly to IQR, we tested the mean, Linear Interpolation, Akima Interpolation, the ARIMA method, time interpolation, correlated futures, and a combination of the last two for MAE. The results of these tests overwhelmingly showed that a combination of time interpolation and correlated futures had the least MAE of all.
Time interpolation was also essential to account for seasonality, as fitting the data over time across several years allowed it to better fit the data’s structure.
How Does AgFlow Handle Seasonality and Market Changes?
Seasonality is an essential mechanism in grains commodities trading and translates directly into the data. We use time interpolation to aggregate the data that will be inputted into the Machine Learning model. The model is then trained and tested on these datasets and can understand the regular pricing changes based on time.
However, structural market changes are impossible to predict by the model as these changes are produced by external factors, such as policy changes, infrastructure changes, or even trade wars and climate change. Nonetheless, the model’s flexibility also presents a remarkable opportunity since it is not based on fixed calendars nor considers the set of rules that governs grain markets. As the forward curves are back-tested every day based on the last year of data, as well as the entire historical set every three months, it adapts to the most recent changes whilst conserving knowledge of seasonality and previous structural changes, if any.
How Does AgFlow Backtest Forward Curves?
The curve backtesting is done on a daily basis. It consists of comparing the predicted values with the actual data and grading how well they matched. The Machine Learning model then refits the data with the recent entries and adapts its calculations according to the grading, ensuring a continuous improvement of the model.
What Are the Differences Between Regular and AgFlow Forward Curves?
Both have perks, limitations, and specific use cases.
AgFlow’s forward curves are complete and give the full picture over a period of 12 months. As such, they are safer to benchmark the MTM process and to calculate VaR and P&L to challenge traders on their positions. Moreover, with forward curves, traders have a more precise tool to detect market signals and predict market movements.
On the other hand, standard forward curves are based on actual prices – whereas AgFlow’s forward curves contain large amounts of artificial prices. Consequently, a trader cannot take positions using our solution. In this case, it acts as a benchmark and is limited by its nature.
Comparison Between Data, Forward curves, and AgFlow’s Forward Curves
Which Tickers Do AgFlow Forward Curves Cover?
There are currently 50 tickers available at launch on the platform, based on the most performant ones.
- Argentina Corn FOB Argentina
- Argentina Feed Barley FOB Argentina
- Argentina Milling Wheat 11.5 FOB Argentina
- Argentina Soymeal CIF Netherlands
- Argentina Soymeal FOB Argentina
- Argentina Soyoil CIF India
- Argentina Soyoil FOB Argentina
- Argentina Sunoil FOB Argentina
- Australia Feed Barley FOB Australia
- Australia Rapeseed FOB Australia
- Baltic Malting Barley FOB Baltic
- Brazil Corn FOB Brazil
- Brazil Soybean CFR China
- Brazil Soybean FOB Brazil
- Brazil Soymeal FOB Brazil
- Brazil Soymeal Pellet FOB Brazil
- Brazil Soyoil FOB Brazil
- Canada Rapeseed 1CDA EXW Canada
- Canada Wheat 1CWAD EXW Canada
- Canada Yellow Peas EXW Canada
- France Corn FOB France
- France Feed Barley FOB France
- France Milling Wheat 11 FOB France
- Germany Rapeseed Meal FOB Germany
- Germany Soybean Oil FOB Germany
- Indonesia Palm Oil FOB Indonesia
- Malaysia Fatty Palm Oil FOB Malaysia
- Malaysia Palm Oil FOB Malaysia
- Malaysia Palm Oil Kernels FOB Malaysia
- Malaysia Palm Olein FOB Malaysia
- Optional Corn CIF Netherlands
- Optional Feed Barley CIF Netherlands
- Optional Palm Oil CFR India
- Optional Palm Olein CFR India
- Optional Soymeal FOB Netherlands
- Optional Soyoil CFR India
- Russia Milling Wheat 12.5 CIF Turkey
- Russia Milling Wheat 12.5 FOB Russia
- Russia Wheat CPT 12.5 Russia
- Spain Wheat Feed EXW
- Ukraine Corn FOB Ukraine
- Ukraine Milling Wheat 11.5 FOB Ukraine
- Ukraine Sunoil CIF India
- Ukraine Sunoil FOB Ukraine
- US Corn FOB US Gulf
- US DDGS CFR China
- US DNS Wheat FOB US Gulf
- US DNS Wheat FOB US PNW
- US Soymeal CFR Bangladesh
- US Soymeal CFR Malaysia
