Add JORA fine-tuning notebook #100
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
| @@ -0,0 +1,802 @@ | |||
| { | |||
There was a problem hiding this comment.
There was a problem hiding this comment.
As stated in the library here
https://github.com/aniquetahir/JORA
| @@ -0,0 +1,802 @@ | |||
| { | |||
There was a problem hiding this comment.
There was a problem hiding this comment.
Not sure if the library is compatible with it but will check
There was a problem hiding this comment.
@windmaple The directory structure needs to be modified manually for the time being until JORA's maintainer updates the repo to include Gemma 2 support
There was a problem hiding this comment.
I'll update the notebook to use Gemma 2 but I figured it wouldn't be ideal as the library hasn't added support for it yet. It works nonetheless, but we need to alter the directory structure for the downloaded Flax model.
|
Also does it crash for you on T4? It keeps crashing due to OOM (CPU) Don't forget to update README/WISHLIST |
I've mentioned in the notebook that it requires an A100 btw. |
The library has no quantization support yet so we're expected to use a high-end GPU for the time being I believe. |
- Add another prompt to test the model
T4 actually works fine. For some reason you need the high RAM instance. Could you try to figure out why? |
Colab using the latest JAX version might be causing issues with the current JORA implementation. I'll investigate and verify if that's really the case. |
|
Even on Kaggle it's hitting Colab's RAM limits (12.7 GB) so I believe it's best to direct free users to run it on Kaggle? @windmaple |
…ilize JAX version - Allow free-tier users to leverage two T4 GPUs on Kaggle - Added the visibility and accessibility of Gemma 2 within the JORA library - Lock the JAX library to the specific version used by Colab to ensure future compatibility and prevent potential issues arising from version discrepancies.
|
Kaggle it is |
3 participants
Update: Switched from Gemma 1.1 to 2