Failed to create Livy session for executing notebook. Error: [BadGateway] The request for workload '

[UdaySutar]

Hi Team,

I'm getting below error while executing notebook activity for scheduled pipeline.can you please explain the reason and how to fix .

• Notebook execution failed at Notebook service with http status code - '200', please check the Run logs on Notebook, additional details - 'Error name - Exception, Error value - Failed to create Livy session for executing notebook. Error: [BadGateway] The request for workload 'SparkCore' of capacity 'AA19707D' failed. HTTP status code: 502.' :

[v-tsaipranay]

Hi @UdaySutar ,

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

[v-tsaipranay]

Hi @UdaySutar ,

Thank you for reaching out to the Microsoft fabric community forum. Also thank you @jennratten for your inputs.

The error “Failed to create Livy session... [BadGateway]” with HTTP status code 502 typically indicates that the Spark runtime couldn’t start due to capacity issues or a transient service error. This may occur if the Spark pool (capacity ‘AA19707D’) was paused, deallocated, or under heavy load at the time of execution.

Please check the Notebook run logs in Synapse or Microsoft Fabric/ADF for more details. Ensure the Spark pool is active, and your workspace has access to available capacity. If you're using Microsoft Fabric, review the Capacity Metrics App to confirm there are sufficient resources (CPU, memory, concurrency).

We also recommend setting a retry policy and increasing the timeout in the pipeline activity to handle transient failures. Let us know if the issue persists after these checks.

I hope this will resolve your issue, if you need any further assistance, feel free to reach out.

If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Thankyou.

[v-tsaipranay]

Hi @UdaySutar ,

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.

Thank you.

[jennratten]

Hello @UdaySutar,

The 502 Bad Gateway error indicates that the request to start a Spark session was routed through a gateway or proxy that received an invalid response from the backend.
The Livy session failure messages means Livy is the service that manages Spark sessions and since it failed, your notebook can't execute Spark code.
The capacity (AA19707D) means that the capacity might be overloaded, misconfigured, or temporarily unavailable.

It's possible that this was a temporary.  I recommend trying to run your notebook again.  If the errors are thrown again, try clearing your browser cache and cookies.

You can check to see which jobs are running and using resources of the same capacity and check capacity utilization by going to the Monitoring Hub or by using the Fabric Capacity Metrics App.