With the increasing demand of e-commerce, the last-mile delivery expenses are currently showing 50 percent or more of the total logistics expenses, especially in the urban cluster regions such as New York and Los Angeles. Existing routing schemes, be they static or dynamic systems of simpler nature, do not keep up with the complexity of the real-time dynamics in urban environments, including congestion, changing density of orders, and unpredictable disruptions. In this paper, a solution based on the integration of Big Data and Deep Reinforcement Learning (DRL) to optimize the path planning and resource scheduling is offered. The model combines real time information on GPS, weather, traffic (through APIs of Waze/Google Maps) and order patterns in the past and uses predictive modeling on LSTM/Transformer networks to predict order density and congestion. The DRA agent serves as the main dispatching system, which adapts routes of vehicles with regard to actual conditions (location, fuel/battery, traffic, package load) and tends to optimize the on-time delivery rates as well as reduce the mileage and operating expenses. The study will seek to offer a self-educative, adaptive solution that will enhance the efficiency of last-mile delivery, which can ultimately change the way the largest carriers and e-commerce giants conduct their dispatch.