|
1 | 1 | # Welcome to the trubrics-sdk |
2 | | -<center> |
3 | 2 |
|
4 | | -[](https://discord.gg/jJ9YDE7qmt) |
5 | | -[<img src="./assets/trubricsdocs.png" width="170">](https://trubrics.github.io/trubrics-sdk/) |
6 | | - |
7 | | -</center> |
8 | | - |
9 | | -<center> |
10 | | - |
11 | | -[](https://github.com/trubrics/trubrics-sdk/actions/workflows/test.yml) |
12 | | -[](https://github.com/trubrics/trubrics-sdk/actions/workflows/docs.yml) |
13 | | -[](https://github.com/trubrics/trubrics-sdk/releases) |
14 | | -[](https://github.com/trubrics/trubrics-sdk/blob/main/LICENSE) |
15 | | - |
16 | | -</center> |
17 | | - |
18 | | -Trubrics enables AI teams to **collect, analyse and manage user prompts & feedback** on models. This allows teams to: |
19 | | - |
20 | | -- **🚨 Identify bugs** - users are constantly running inference on models, and may be more likely to find bugs than an ML monitoring system |
21 | | -- **🧑💻️ Fine tune** - users often hold domain knowledge that can be useful to fine tune models |
22 | | -- **👥 Align** - identifying user preferences will help to align models to users |
23 | | - |
24 | | -<img src="./assets/trubrics-example.png" width="800"> |
25 | | - |
26 | | -## Try our LLM demo |
27 | | - |
28 | | -1. Create your **free account**: |
29 | | - |
30 | | - [<img src="./assets/sign_up.png" width="200">](https://trubrics.streamlit.app/) |
31 | | - |
32 | | -2. Save feedback to Trubrics from our **demo LLM app**: |
33 | | - |
34 | | - [<img src="https://static.streamlit.io/badges/streamlit_badge_black_white.svg" width="200">](https://trubrics-llm-example-chatbot.streamlit.app/) |
35 | | - |
36 | | -Or watch a step by step video of integrating Trubrics into the LLM Streamlit app [here](https://www.youtube.com/watch?v=2Qt54qGwIdQ). |
37 | | - |
38 | | -## Collect user prompts & feedback with the Python SDK |
39 | | - |
40 | | -The python SDK allows you to collect user prompts & feedback from your ML apps from any python backend or web framework. Install it with: |
41 | | - |
42 | | -```console |
43 | | -pip install trubrics |
44 | | -``` |
45 | | - |
46 | | -Now set your [Trubrics](https://trubrics.streamlit.app/) `email` and `password` as environment variables: |
47 | | - |
48 | | -```bash |
49 | | -export TRUBRICS_EMAIL="trubrics_email" |
50 | | -export TRUBRICS_PASSWORD="trubrics_password" |
51 | | -``` |
52 | | - |
53 | | -and push some user prompts & feedback to the `default` project & feedback component: |
54 | | - |
55 | | -```python |
56 | | -import os |
57 | | -from trubrics import Trubrics |
58 | | - |
59 | | -trubrics = Trubrics( |
60 | | - project="default", |
61 | | - email=os.environ["TRUBRICS_EMAIL"], |
62 | | - password=os.environ["TRUBRICS_PASSWORD"], |
63 | | -) |
64 | | - |
65 | | -user_prompt = trubrics.log_prompt( |
66 | | - config_model={"model": "gpt-3.5-turbo"}, |
67 | | - prompt="Tell me a joke", |
68 | | - generation="Why did the chicken cross the road? To get to the other side.", |
69 | | -) |
70 | | - |
71 | | -user_feedback = trubrics.log_feedback( |
72 | | - component="default", |
73 | | - model=user_prompt.config_model.model, |
74 | | - prompt_id=user_prompt.id, |
75 | | - user_response={ |
76 | | - "type": "thumbs", |
77 | | - "score": "👎", |
78 | | - "text": "Not a very funny joke...", |
79 | | - } |
80 | | -) |
81 | | -``` |
82 | | - |
83 | | -## Collect user prompts & feedback from a Streamlit app |
84 | | - |
85 | | -To start collecting user feedback from your [Streamlit](https://streamlit.io/) app, install the additional dependency: |
86 | | - |
87 | | -```console |
88 | | -pip install "trubrics[streamlit]" |
89 | | -``` |
90 | | - |
91 | | -and test this code snippet in your app: |
92 | | - |
93 | | -```python |
94 | | -import streamlit as st |
95 | | -from trubrics.integrations.streamlit import FeedbackCollector |
96 | | - |
97 | | -collector = FeedbackCollector( |
98 | | - email=st.secrets.TRUBRICS_EMAIL, |
99 | | - password=st.secrets.TRUBRICS_PASSWORD, |
100 | | - project="default" |
101 | | -) |
102 | | - |
103 | | -user_feedback = collector.st_feedback( |
104 | | - component="default", |
105 | | - feedback_type="thumbs", |
106 | | - open_feedback_label="[Optional] Provide additional feedback", |
107 | | - model="gpt-3.5-turbo", |
108 | | - prompt_id=None, # checkout collector.log_prompt() to log your user prompts |
109 | | -) |
110 | | - |
111 | | -if user_feedback: |
112 | | - st.write("#### Raw feedback saved to Trubrics:") |
113 | | - st.write(user_feedback) |
114 | | -``` |
115 | | - |
116 | | -For a full examples logging user prompts and feedback in Streamlit, see our [Streamlit integration docs](https://trubrics.github.io/trubrics-sdk/integrations/streamlit/). |
117 | | - |
118 | | -## Collect user feedback from a React.js app |
119 | | - |
120 | | -To collect user feedback from a React application, check out [this example](https://github.com/trubrics/trubrics-sdk/tree/main/examples/react_js). |
121 | | - |
122 | | -## What's next? |
123 | | - |
124 | | -- If you haven't already, create a free account or sign in to [Trubrics](https://trubrics.streamlit.app/). |
125 | | -- Get more technical information from our [docs](https://trubrics.github.io/trubrics-sdk/): |
126 | | - - Collect & analyse [user prompts](https://trubrics.github.io/trubrics-sdk/platform/user_prompts/) |
127 | | - - Collect & analyse [user feedback](https://trubrics.github.io/trubrics-sdk/platform/user_feedback/) |
128 | | - - Manage user feedback with [Issues](https://trubrics.github.io/trubrics-sdk/platform/issues/) |
129 | | -- Check out our [website](https://www.trubrics.com/home) for more information about Trubrics. |
| 3 | +> [!IMPORTANT] |
| 4 | +> We are working on a new analytics platform, and we're looking for beta testers. |
| 5 | +> Please [reach out](https://calendly.com/joel-hodgson-1/30min?month=2024-05) to learn more. |
0 commit comments