Title : Diesel blend optimization using value-based deep reinforcement learning agent
Abstract:
The present scenario of optimization of product stream blending in HPCL refineries is a manual process managed by the refinery operations planning team. This workflow relies heavily on periodic laboratory results and requires extensive, continuous coordination across multiple refinery units. Because the process is manual, reactive, and time-intensive, there is a substantial margin for human error and missed optimization opportunities, often resulting in costly product quality giveaway (QGA).
To resolve these inefficiencies, an in-house, value-based Reinforcement Learning (RL) agent has been developed. The RL agent dynamically evaluates operational states, and latest available laboratory analysis data and sends real-time optimization targeted recommendations across the plant. This includes primary and secondary level units such as Crude Distillation Units (CDUs), Fluid Catalytic Cracking Units (FCCUs), and tertiary treatment units such as diesel hydrotreaters, and full-conversion hydrocrackers. The system provides real-time forecasted properties of the product tanks and equips operations planners with an intuitive interface to directly compare projected tank compositions under two scenarios: with the RL recommendations implemented versus without it.
Under the hood, the agent is essentially a Deep Q-Network which is trained using historical scenario data. The underlying model is trained to maximize long-term rewards by choosing best course of action for various values of states. This way, it can realign itself dynamically under different conditions to avoid off-spec products while reducing the QGA with minimum product routing changes.
By transitioning to a proactive, RL-driven blending strategy, the operations team is now empowered to make data-backed decisions instantly. The solution is presently built for handling diesel optimization across both HPCL Refineries but can be extended to other blends. In the longer run, it is expected to streamline inter-unit coordination and reduce the Quality Giveaway (QGA) of product diesel tanks by at least 50%, driving significant economic value and operational efficiency.
