Google Summer of Code 2025: Cloudcasting ML Discussion Thread

[dfulu]

Cloudcasting ML Discussion Thread

This space is for you to ask any questions you have about this project. We're here to provide clarifications and help you understand the project's goals, scope, and requirements. Feel free to ask about anything that interests you!

Please note that this discussion is for questions and clarifications, not for formal applications.

Project Description

Traditionally, forecasting tools are trained using historical satellite data. As part of an innovative new project, we have been training a model to predict satellite images up to 3 hours ahead over the UK. This work in early stages, but we have already proved that a satellite forecast using a very simple ML model can improve our solar energy forecast. There is lots of opportunity to improve on this new and unique satellite forecast, from trying different video prediction and AI weather model architectures to training a diffusion based model to stop the satellite forecast being blurry.

Expected Outcome

An improved satellite forecast model

Other Key Information

• Expected Size: 350hrs
• Skills: ML, pytorch, python
• Difficulty level: Hard
• Related Reading:
  • https://www.openclimatefix.org/projects/cloud-forecasting
  • https://github.com/alan-turing-institute/cloudcasting
  • https://github.com/openclimatefix/sat_pred
• Potential mentors: @dfulu

[sumana-2705]

Hello @dfulu

I have opened an issue regarding training models on CPU-only machines. I wanted to ask whether GPU acceleration is strictly necessary for training, or if the models can be trained efficiently on a CPU as well. Are there any recommended optimizations or modifications to make training feasible on CPU-only systems?

[dfulu]

Hi @sumana-2705, I've responded to your issue on that repo. I haven't tested that code without using a GPU though, so you may encounter more issues.

More generally, even if the code can be run on CPU I think it will turn out to be too slow to train it this way. The current best performing model that we've trained (the simVP model) was trained on a GPU (though only an single GPU and not a great one) for 12 days. The model performance was improving for pretty much all of those 12 days of training. I'd expect CPU training to take significantly longer

[aayushyatiwari]

model) was trained on a GPU (though only an single GPU and not a great one) for 12 days. The model performance was improving for pretty much all of those 12 days of training

hey! uh dam, this might sound like a stupid question but I don't know. when training for 12 days, what machines did you use? were they normal pcs with NVIDIA GPUs or where they modified machines? Because I have a 4050 RTX and I was thiking to work on this project myself.

[dfulu]

I was a fairly modest GPU - an NVIDIA TESLA T4

[Syedmahmood777]

Hey @dfulu,

My name is Syed Mahmood, and I am a CSE-DS professional with a strong background in machine learning. The Cloudcasting ML project really interests me, and I would love to contribute to it as part of GSoC '25. I wanted to ask if there are any subject-specific resources or key areas I should focus on to better understand this project and align my contributions effectively.

I have experience working with PyTorch and ML model development, so I’d love to know if there are any particular frameworks, datasets, or methodologies I should get familiar with beforehand. Looking forward to your guidance!

[dfulu]

Hi @Syedmahmood777, thanks for your interest.

This project is still in its early stages, and we're primarily working with a custom dataset of satellite imagery which we maintain as a google public dataset. The focus is on developing machine learning models that predict future satellite images based on past observations—essentially a video-to-video prediction task.

Relevant areas to explore would be spatiotemporal deep learning models, such as ConvLSTMs, Transformers for video prediction, and/or diffusion models for reducing blur in predicted frames.

A good starting point might be to review our the github repo here, which includes the code for preparing and downloading datasets, and also links to a few of our ML experiment repos for this project. You might also find it helpful to look into research on deep learning for weather forecasting and video prediction models.

[abdksyed]

Hi James,

First, came to know about Open Climate Fix, and wanted to give a huge round of applause for building the need of hour tech for climate change. I have previously worked with image and video data, but mainly with medical data. Before I commit, wanted to know is there a need for subject expertise like GIS, Climate Science etc.?

You also mentioned that this project is in it's early stage, but is there any similar published word out there? The repo (and other linked repos) you shared doesn't have any detailed information of the models, results or anything as such.

Thanks!

[dfulu]

Hi @abdksyed, there is no expectation of knowledge of GIS or climate science. Our tech stack is all python and pytorch and we are mostly treating this as a video-to-video prediction task. Knowledge of weather/climate is not key but could be a bit of an advantage to help design better networks.

We have not published anything about this work yet, but a couple of the papers which have been of interest to us are [1] and [2]

• [1] https://arxiv.org/abs/2206.05099
• [2] Earthformer: Exploring Space-Time Transformers for Earth System Forecasting

[abdksyed]

Oh!, I realised that this project is part of something called GSoC, which I am not 100% sure is for students. I am a full-time AI software engineer, who wanted to contribute to a good OSS project, and came across this project.

I don't know if you are the right person to ask this, but will I have to apply for GSoC to work on this project? (as far as I have seen the eligibility doesn't mention that applicants have to be students, it just mentioned that one have to at least 18 years of age and a resident in participating county, and I satisfy both of them.)
But if GSoC is only for students, is there any way for me to be part of this project? I wanted to work on some nice CV OSS projects in a community setting

[dfulu]

I believe you can apply to GSoC if you are new to contributing to open source code, you do not need to be a student. At the moment we are only thinking of this project as part of GSoC.

[vinay752]

Hi James,

I'm Vinay Palakurthy. I hope you are doing great! I've gone through Open Climate Fix's website and read the story of OCF. I felt happy reading the origin of Open Climate Fix, developing AI driven solutions to improve efficiency in the energy sector and reducing green house gas emissions.

I was particularly impressed by the 5% accuracy improvement in the UK in solar generation forecast with AI cloud Forecasting that out-performed both UK and Europe Met Office Weather Services' when measuring the impact on short-term solar generation forecasting. It's amazing what Open Climate Fix has achieved in 6 years!

I'm excited for Cloudcasting to go live in the summer 2025. I'm eager to contribute to the Cloudcasting project, especially given my background in data science and with the expertise in time series forecasting. I believe my skills and knowledge in these areas could be valuable to your team.

I have couple of question for you if you don't mind to answer:

1. May I know how much of data did your team use to train the models to beat the agencies like Met Office Weather Services' who are in the market for a really long time?
2. Do you see Cloudcasting going beyond solar,generation forecast in the future, like maybe wind for those big windmills?

Best,
Vinay

[dfulu]

Hi @vinay752, thanks for your question

1. For training the cloudcasting model we use 15-minutely satellite data covering 2008-2022. Since we slice it down to only cover the UK it is about 1TB.

   This is just for the cloudcasting model which predicts future satellite data. We have a second model which ingests satellite data, the output of the cloudcasting model, and multiple weather forecasts from the Met Office and ECMWF. These add up to about maybe 10TB, but won't be used as part of this project.

2. Not at the moment. It isn't clear to us whether satellite data is useful for wind forecasting - we haven't tested it

[siddharth7113]

Hey @dfulu

First off, thanks for making this discussion space available! I’ve been going through the repo and had a few questions:

1. Last week, I set up the repo and downloaded the data from the public bucket. It took me nearly 18 hours to download just the 2019 data—probably due to a slow internet connection. Is there any plan to integrate this with ocf-data-sampler so that we can train on a larger dataset directly from cloud storage? It would be great if we could create chunks of Torch datasets on the fly and train models without needing to download everything locally. Apologies if I’m missing something here!

2. I also went through the Earthformer paper. With this project, is the main goal to develop new architectures that perform better on the current dataset, or is there also scope for increasing the dataset size to train models on a larger scale?

3. I noticed you mentioned AI diffusion models (video to video) in a previous message. Are there any recent papers in this area that you’d recommend checking out, Also are GANs a potentiall candidate for this , or do they have some inherent flaws with this?

4. Given the high computational cost of diffusion models, would most of the training be done on the cloud? Or are there plans to optimize it for local GPUs as well?

[dfulu]

Hi @siddharth7113, thanks for your qustions

1. The reason we download in the first place is because having the data stored locally stops the network speed from slowing down training. Streaming samples directly from cloud storage requires even faster network speed than storing them locally. So no, we don't plan to integrate with ocf-data-sampler. When downloading, I recommend using the utilities in https://github.com/alan-turing-institute/cloudcasting which crops the images down to the UK and subsets them to 1 image every 15 mins. The original images cover a much larger area and are 5-minutely

2. The main aim of the project is to make the model perform better on the current dataset. We believe there is a lot of room for improvement for our UK solar forecast we only really want to cover the UK currently. We will likely scale up geographically, but not until we've spent time making the model moire accurate at a smaller scale.

3. I don't have an particular papers to mention. Some if my colleagues found this helpful: https://arxiv.org/abs/2206.00364
   We don't plan to use GANs. As I understand it, now that diffusion networks have been developed more, they have made GANs outdated

4. As it is, it should be possible to train on a reasonably powered desktop. We found that about 1TB of disk space, one NVIDIA T4 GPU (with 16GB VRAM), and a CPU with 8 cores and 30GB RAM was enough to train a model. However, last year for our computationally demanding GSoC projects we set up candidates to use OCF's internal computer or on cloud VMs

[ShubhamChauhan22222]

Hi @dfulu, I am a 3rd-year student, pursuing an Integrated M.Tech in Mathematics and Computing. I have a strong background in AI/ML and GPU computing, and I am very interested in contributing to this project.
I have researched the available data in Zarr format and explored previous work related to satellite forecasting and the Cloudcasting repo. Now, I would like to understand the specific models and techniques you plan to implement for improving the satellite forecast model. If you have any particular approaches in mind, I’d be happy to explore them, or I can experiment with different methods independently.
Due to resource and time constraints, I will initially work with 30 days of data and share my findings here. Additionally, is there a way to connect privately for further discussion?
Looking forward to your guidance and feedback!

[dfulu]

Hi @ShubhamChauhan22222, thanks for your question

This is intended to be quite an open ended project. We don't have specific models and techniques in mind yet. So far the best model we have tried is SimVP, which is a very simple video prediction model. There are papers which follow on from SimVP to improve it. We have not fully explored those, but that would be my first intended direction. There are plenty of shortcomings in the SimVP architecture which we could improve on. We also tried using training the Earthformer model, but with the default parameter settings we could not beat SimVP for this task. The project will likely be to improve on the performance of SimVP, but a student with a strong interest in another model could try it out.

We will not be meeting privately with students until the project begins