All files deleted from job

[BiJoe]

Suddenly, after attempting to resume a job, all files are gone without any notice or warning. 

[nsprudhvi · Accepted solution]

Hi @BiJoe,

It looks like you may have accidentally deleted a DAG from the Airflow UI while in "Monitor in Apache Airflow." This can happen if the UI shows an option that allows you to remove DAGs without explicit warnings.

If you haven't backed up your DAG files, you might need to recreate them. Going forward, it's a good idea to have a version control system or backups in place to prevent data loss.

Let me know if you need help recovering or recreating your DAGs!

Best regards,
Sai Prudhvi Neelakantam
Data Engineer | 3x Microsoft Certified

💼 LinkedIn: in/saiprudhvineelakantam | 🌐 geekindata.com

If this helps, kindly mark this as the accepted solution! 👍

[Anonymous]

Hi @BiJoe 

Here are some of my suggestions:

Try recreating and test to see if the problem still exists.

Make sure Airflow is properly connected to its metadata database. If there is a problem with the connection, it may cause the DAG and plug-ins to fail to display.

Verify that the DAG and plug-in files are still in the correct directory and have not been moved or deleted. If files are moved or deleted, Airflow will not be able to find them.

Ensure that Airflow has the necessary read permissions on the DAG and the plug-in directory. Airflow may not be able to load these files if the permissions are insufficient.

If there have been any recent changes to Airflow configuration, they may affect the visibility of the DAG and plug-in. Check the configuration file to make sure all Settings are correct.

Regards,

Nono Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.