add docs for LLM explanation for OpenAI library

Author: lopuhin

README.rst

@@ -54,6 +54,7 @@ It provides support for the following machine learning frameworks and packages:
 * sklearn-crfsuite_. ELI5 allows to check weights of sklearn_crfsuite.CRF
   models.
 
+* OpenAI_ python client. ELI5 allows to explain LLM predictions with token probabilities.
 
 ELI5 also implements several algorithms for inspecting black-box models
 (see `Inspecting Black-Box Estimators`_):
@@ -81,6 +82,7 @@ and formatting on a client.
 .. _Catboost: https://github.com/catboost/catboost
 .. _Permutation importance: https://eli5.readthedocs.io/en/latest/blackbox/permutation_importance.html
 .. _Inspecting Black-Box Estimators: https://eli5.readthedocs.io/en/latest/blackbox/index.html
+.. _OpenAI: https://github.com/openai/openai-python
 
 License is MIT.
 

docs/source/libraries/index.rst

@@ -12,5 +12,5 @@ Supported Libraries
    catboost
    lightning
    sklearn_crfsuite
+   openai
    keras
-

docs/source/libraries/openai.rst

@@ -0,0 +1,63 @@
+.. _library-openai:
+
+OpenAI
+======
+
+OpenAI_ provides a client library for calling Large Language Models (LLMs).
+
+.. _OpenAI: https://github.com/openai/openai-python
+
+eli5 supports :func:`eli5.explain_prediction` for
+``ChatCompletion``, ``ChoiceLogprobs`` and ``openai.Client`` objects,
+highlighting tokens proportionally to the log probability.
+More likely tokens are highligted in green,
+while unlikely tokens are highlighted in red.
+
+Explaining with a client, invoking the model with ``logprobs`` enabled:
+::
+
+    import eli5
+    import opeanai
+    client = openai.Client()
+    prompt = 'some string'  # or [{"role": "user", "content": "some string"}]
+    explanation = eli5.explain_prediction(client, prompt, model='gpt-4o')
+    explanation
+
+You may pass any extra keyword arguments to :func:`eli5.explain_prediction`,
+they would be passed to the ``client.chat.completions.create``,
+e.g. you may pass ``n=2`` to get multiple responses
+and see explanations for each of them.
+
+You'd normally want to run it in a Jupyter notebook to see the explanation
+formatted as HTML.
+
+You can access the ``Choice`` object on the ``explanation.targets[0].target``:
+::
+
+    explanation.targets[0].target.message.content
+
+If you have already obtained a chat completion with ``logprobs`` from OpenAI client,
+you may call :func:`eli5.explain_prediction` with
+``ChatCompletion`` or ``ChoiceLogprobs`` like this:
+::
+
+    chat_completion = client.chat.completions.create(
+        messages=[{"role": "user", "content": prompt}],
+        model="gpt-4o",
+        logprobs=True,
+    )
+    eli5.explain_prediction(chat_completion)  # or
+    eli5.explain_prediction(chat_completion.choices[0].logprobs)
+
+.. note::
+    While token probabilities reflect model uncertainty in many cases,
+    they are not always indicative,
+    e.g. in case of `Chain of Thought <https://arxiv.org/abs/2201.11903>`_
+    preceding the final response.
+
+.. note::
+    Top-level :func:`eli5.explain_prediction` calls are dispatched
+    to :func:`eli5.llm.explain_prediction.explain_prediction_openai_client`
+    or :func:`eli5.llm.explain_prediction.explain_prediction_openai_completion`
+    or :func:`eli5.llm.explain_prediction.explain_prediction_openai_logprobs`
+    .

docs/source/overview.rst

@@ -50,6 +50,8 @@ following machine learning frameworks and packages:
 * :ref:`library-sklearn-crfsuite`. ELI5 allows to check weights of
   sklearn_crfsuite.CRF models.
 
+* :ref:`library-openai`. ELI5 allows to explain LLM predictions with token probabilities.
+
 ELI5 also implements several algorithms for inspecting black-box models
 (see :ref:`eli5-black-box`):