8b60bc57e6
simplify LLM's job. Do not request Json output with a single key. Instead, make sure LLM don't output any extra information. By simplifying LLM's job, we're making sure its output can be parsed. I did a quick test with the Translate prompt. Adding instructions to output only translated text seems enough after a bunch of tests. I did a small prompt engineering, using ChatGPT and Claude to generate a proper system prompt … it works quite okay BUT there is room for improvement for sure. I'ven't searched yet OS prompts we could find in a prompt library. Perfect translation job seems to be a difficult job for a 8B model. Please note I haven't updated yet the other prompts, let's discuss it before. I ran my experiment with our internal LLM which is optimized for throughput, and not latency (there is a trade-off). I'll try fine tune few of its parameters to see if I can reduce its latency. For 880 tokens (based on chatgpt tokens counter online). It takes roughly 17s, vs ~40s for Albert CNRS 70B. For 180 tokens it takes roughly 3s. Without a proper UX (eg. a nicer loading animation, streaming tokens) it feels a decade. However, asking Chatgpt the same job take the same amount, from submitting the request to the last token being generated.
Impress
Impress is a web application for real-time collaborative text editing with user and role based access rights. Features include :
- User authentication through OIDC
- BlocNote.js text editing experience (markdown support, dynamic conversion, block structure, slash commands for block creation)
- Document export to pdf and docx from predefined templates
- Granular document permissions
- Public link sharing
- Offline mode
Impress is built on top of Django Rest Framework, Next.js and BlocNote.js
Getting started
Prerequisite
Make sure you have a recent version of Docker and Docker Compose installed on your laptop:
$ docker -v
Docker version 20.10.2, build 2291f61
$ docker compose -v
docker compose version 1.27.4, build 40524192
⚠️ You may need to run the following commands with
sudobut this can be avoided by assigning your user to thedockergroup.
Project bootstrap
The easiest way to start working on the project is to use GNU Make:
$ make bootstrap FLUSH_ARGS='--no-input'
This command builds the app container, installs dependencies, performs
database migrations and compile translations. It's a good idea to use this
command each time you are pulling code from the project repository to avoid
dependency-releated or migration-releated issues.
Your Docker services should now be up and running 🎉
You can access to the project by going to http://localhost:3000. You will be prompted to log in, the default credentials are:
username: impress
password: impress
📝 Note that if you need to run them afterwards, you can use the eponym Make rule:
$ make run-with-frontend
⚠️ For the frontend developper, it is often better to run the frontend in development mode locally. To do so, install the frontend dependencies with the following command:
$ make frontend-install
And run the frontend locally in development mode with the following command:
$ make run-frontend-development
To start all the services, except the frontend container, you can use the following command:
$ make run
Adding content
You can create a basic demo site by running:
$ make demo
Finally, you can check all available Make rules using:
$ make help
Django admin
You can access the Django admin site at http://localhost:8071/admin.
You first need to create a superuser account:
$ make superuser
Contributing
This project is intended to be community-driven, so please, do not hesitate to get in touch if you have any question related to our implementation or design decisions.
License
This work is released under the MIT License (see LICENSE).