Compare commits

..

24 Commits

Author SHA1 Message Date
Jacques ROUSSEL
ccc206113b overwrite commit 2024-10-18 11:36:29 +02:00
Anthony LC
133b005263 change model IA 2024-10-18 11:23:04 +02:00
Anthony LC
ff845e1a6b fixup increase throttle IA 2024-10-17 17:09:38 +02:00
Anthony LC
980605aecd increase throttle IA 2024-10-17 16:34:25 +02:00
lebaudantoine
f72547e6e6 fix tag 2024-10-17 16:23:02 +02:00
lebaudantoine
3310d8b18f update tag 2024-10-17 16:23:02 +02:00
lebaudantoine
e0d41b712e wip expose a summary 2024-10-17 16:23:02 +02:00
Jacques ROUSSEL
2f8c50540e 🚀(docs-ia) deploy new environment
Because we use staging for German people, we need a new environment for
our demo
2024-10-17 16:23:02 +02:00
Anthony LC
e816f0afc8 🚸(frontend) add toast error when AI request fails
When the AI request fails, a toast error is
displayed to the user.
2024-10-17 16:22:13 +02:00
lindenb1
7e8732822b 🐛(docker) update docker-compose.yml to make nginx depend on app-dev
Modified docker-compose.yml to ensure nginx starts only after app-dev.

Signed-off-by: lindenb1 <linden@b1-systems.de>
2024-10-17 14:50:21 +02:00
Anthony LC
9ed6b11bb1 ⬆️(dependencies) update js dependencies 2024-10-17 13:11:01 +02:00
Anthony LC
62124ae475 🌐(frontend) translate last features
Translate:
- IA Buttons
- doc clipboard markdown / html
2024-10-17 10:11:37 +02:00
Anthony LC
c327928921 🐛(frontend) fix flaky e2e test
A test on e2e was flaky, this commit fixes it.
2024-10-16 22:58:52 +02:00
Anthony LC
be26a9457f 🔧(helm) add ai setting to environments
Add the ai setting to the environments.
2024-10-16 22:58:52 +02:00
Anthony LC
5dc43cbc8b (frontend) add ai blocknote feature
Add AI button to the editor toolbar.
We can use AI to generate content with our editor.
A list of predefined actions are available to use.
2024-10-16 22:58:52 +02:00
Anthony LC
9abf6888aa 🎨(frontend) reduce prop drilling thanks to doc store
We start to have a deep prop drilling with doc,
time to use the doc store to reduce that.
We still prefer to pass the doc as a prop to
keep our component as "pure" as possible, but if
the drilling is too deep, better
to use the doc store.
2024-10-16 22:58:52 +02:00
Anthony LC
aff3b43c9d (backend) create ai endpoint
We created 2 new action endpoints on the document
to perform AI operations:
- POST /api/v1.0/documents/{uuid}/ai-transform
- POST /api/v1.0/documents/{uuid}/ai-translate
2024-10-16 22:58:52 +02:00
Samuel Paccoud - DINUM
e8d95facdf (backend) allow uploading more types of attachments
We want to allow users to upload files to a document, not just images.
We try to enforce coherence between the file extension and the real
mime type of its content. If a file is deemed unsafe, it is still accepted
during upload and the information is stored as metadata on the object
for display to readers.
2024-10-16 19:40:28 +02:00
Samuel Paccoud - DINUM
a9f08df566 (backend) move freezegun to dev dependencies
Freezegun is for testing and should not be installed in the
production image.
2024-10-16 19:40:28 +02:00
Samuel Paccoud - DINUM
2fecbc1162 🚚(backend) split test file for api template accesses
The number of lines in this file had exceeded 1000 lines.
2024-10-16 19:16:50 +02:00
Samuel Paccoud - DINUM
1fc3029d12 🐛(backend) fix dysfunctional permissions on document create
When creating a document access, users were benefitting on the targeted
document from the highest access right they have among all documents.
This is because we forgot to filter on the document ID when retrieving
the role of the user. We improved all tests to secure this issue.
2024-10-16 19:16:50 +02:00
rvveber
bbcb5e0cf1 (frontend) added copy-as buttons for HTML and Markdown
Add buttons to copy editor content as HTML or Markdown. Closes #300
2024-10-16 17:57:10 +02:00
renovate[bot]
e4a7ac0f3c ⬆️(dependencies) update python dependencies 2024-10-16 10:41:25 +02:00
Anthony LC
24630791d8 ♻️(email) use full name instead of email
If the full name is available,
we will use it to identify the user in the email
instead of the email address.
2024-10-16 09:36:33 +02:00
75 changed files with 5568 additions and 2588 deletions

View File

@@ -4,6 +4,7 @@ creation_rules:
- age:
- age15fyxdwmg5mvldtqqus87xspuws2u0cpvwheehrtvkexj4tnsqqysw6re2x # jacques
- age16hnlml8yv4ynwy0seer57g8qww075crd0g7nsundz3pj4wk7m3vqftszg7 # github-repo
- age1qy04neuzwpasmvljqrcvhwnf0kz5cpyteze38c8avp0czewskasszv9pyw # argocd
- age1plkp8td6zzfcavjusmsfrlk54t9vn8jjxm8zaz7cmnr7kzl2nfnsd54hwg # Anthony Le-Courric
- age12g6f5fse25tgrwweleh4jls3qs52hey2edh759smulwmk5lnzadslu2cp3 # Antoine Lebaud
- age1hnhuzj96ktkhpyygvmz0x9h8mfvssz7ss6emmukags644mdhf4msajk93r # Samuel Paccoud

View File

@@ -9,15 +9,23 @@ and this project adheres to
## [Unreleased]
## Added
- ✨AI to doc editor #250
- ✨(backend) allow uploading more types of attachments #309
- ✨(frontend) add buttons to copy document to clipboard as HTML/Markdown #300
## Changed
- ♻️(frontend) More multi theme friendly #325
- ♻️ Bootstrap frontend #257
- ♻️ Add username in email #314
## Fixed
🐛(frontend) invalidate queries after removing user #336
- 🐛(frontend) invalidate queries after removing user #336
- 🐛(backend) Fix dysfunctional permissions on document create #329
- 🐛(backend) fix nginx docker container #340
## [1.5.1] - 2024-10-10
@@ -25,7 +33,6 @@ and this project adheres to
- 🐛(db) fix users duplicate #316
## [1.5.0] - 2024-10-09
## Added

View File

@@ -13,9 +13,6 @@ RUN apk update && \
# ---- Back-end builder image ----
FROM base AS back-builder
RUN apk add \
cargo
WORKDIR /builder
# Copy required python dependencies
@@ -68,14 +65,15 @@ ENV PYTHONUNBUFFERED=1
# Install required system libs
RUN apk add \
gettext \
cairo \
libffi-dev \
gdk-pixbuf \
pango \
pandoc \
font-noto-emoji \
file \
font-noto \
font-noto-emoji \
gettext \
gdk-pixbuf \
libffi-dev \
pandoc \
pango \
shared-mime-info
# Copy entrypoint

View File

@@ -122,6 +122,7 @@ logs: ## display app-dev logs (follow mode)
run: ## start the wsgi (production) and development server
@$(COMPOSE) up --force-recreate -d celery-dev
@$(COMPOSE) up --force-recreate -d nginx
@$(COMPOSE) up --force-recreate -d y-provider
@echo "Wait for postgresql to be up..."
@$(WAIT_DB)

View File

@@ -63,7 +63,6 @@ services:
- mailcatcher
- redis
- createbuckets
- nginx
celery-dev:
user: ${DOCKER_USER:-1000}
@@ -118,6 +117,7 @@ services:
- ./docker/files/etc/nginx/conf.d:/etc/nginx/conf.d:ro
depends_on:
- keycloak
- app-dev
frontend-dev:
user: "${DOCKER_USER:-1000}"

View File

@@ -39,3 +39,8 @@ LOGOUT_REDIRECT_URL=http://localhost:3000
OIDC_REDIRECT_ALLOWED_HOSTS=["http://localhost:8083", "http://localhost:3000"]
OIDC_AUTH_REQUEST_EXTRA_PARAMS={"acr_values": "eidas1"}
# AI
AI_BASE_URL=https://openaiendpoint.com
AI_API_KEY=password
AI_MODEL=llama

Submodule secrets updated: 38594182e8...59590c285c

View File

@@ -8,6 +8,8 @@ from rest_framework import views as drf_views
from rest_framework.decorators import api_view
from rest_framework.response import Response
from ..models import Document, User, RoleChoices, DocumentAccess
def exception_handler(exc, context):
"""Handle Django ValidationError as an accepted exception.
@@ -38,3 +40,32 @@ def get_frontend_configuration(request):
}
frontend_configuration.update(settings.FRONTEND_CONFIGURATION)
return Response(frontend_configuration)
@api_view(["POST"])
def create_summary(request):
"""Wip."""
data = request.data
document = Document(
title="Votre résumé",
link_reach="authenticated",
link_role="reader",
)
document.save()
owner_user = User.objects.get(email=data["owner"])
document_access = DocumentAccess(
user=owner_user,
document=document,
role=RoleChoices.OWNER
)
document_access.save()
document.content = data["content"]
document.save()
return Response({"id": document.id})

View File

@@ -6,9 +6,11 @@ from django.conf import settings
from django.db.models import Q
from django.utils.translation import gettext_lazy as _
import magic
from rest_framework import exceptions, serializers
from core import models
from core import enums, models
from core.services.ai_services import AI_ACTIONS
class UserSerializer(serializers.ModelSerializer):
@@ -69,6 +71,7 @@ class BaseAccessSerializer(serializers.ModelSerializer):
if not self.Meta.model.objects.filter( # pylint: disable=no-member
Q(user=user) | Q(team__in=user.teams),
role__in=[models.RoleChoices.OWNER, models.RoleChoices.ADMIN],
**{self.Meta.resource_field_name: resource_id}, # pylint: disable=no-member
).exists():
raise exceptions.PermissionDenied(
"You are not allowed to manage accesses for this resource."
@@ -218,16 +221,42 @@ class FileUploadSerializer(serializers.Serializer):
f"File size exceeds the maximum limit of {max_size:d} MB."
)
# Validate file type
mime_type, _ = mimetypes.guess_type(file.name)
if mime_type not in settings.DOCUMENT_IMAGE_ALLOWED_MIME_TYPES:
mime_types = ", ".join(settings.DOCUMENT_IMAGE_ALLOWED_MIME_TYPES)
raise serializers.ValidationError(
f"File type '{mime_type:s}' is not allowed. Allowed types are: {mime_types:s}"
)
extension = file.name.rpartition(".")[-1] if "." in file.name else None
# Read the first few bytes to determine the MIME type accurately
mime = magic.Magic(mime=True)
magic_mime_type = mime.from_buffer(file.read(1024))
file.seek(0) # Reset file pointer to the beginning after reading
self.context["is_unsafe"] = (
magic_mime_type in settings.DOCUMENT_UNSAFE_MIME_TYPES
)
extension_mime_type, _ = mimetypes.guess_type(file.name)
# Try guessing a coherent extension from the mimetype
if extension_mime_type != magic_mime_type:
self.context["is_unsafe"] = True
guessed_ext = mimetypes.guess_extension(magic_mime_type)
# Missing extensions or extensions longer than 5 characters (it's as long as an extension
# can be) are replaced by the extension we eventually guessed from mimetype.
if (extension is None or len(extension) > 5) and guessed_ext:
extension = guessed_ext[1:]
if extension is None:
raise serializers.ValidationError("Could not determine file extension.")
self.context["expected_extension"] = extension
return file
def validate(self, attrs):
"""Override validate to add the computed extension to validated_data."""
attrs["expected_extension"] = self.context["expected_extension"]
attrs["is_unsafe"] = self.context["is_unsafe"]
return attrs
class TemplateSerializer(BaseResourceSerializer):
"""Serialize templates."""
@@ -350,3 +379,33 @@ class VersionFilterSerializer(serializers.Serializer):
page_size = serializers.IntegerField(
required=False, min_value=1, max_value=50, default=20
)
class AITransformSerializer(serializers.Serializer):
"""Serializer for AI transform requests."""
action = serializers.ChoiceField(choices=AI_ACTIONS, required=True)
text = serializers.CharField(required=True)
def validate_text(self, value):
"""Ensure the text field is not empty."""
if len(value.strip()) == 0:
raise serializers.ValidationError("Text field cannot be empty.")
return value
class AITranslateSerializer(serializers.Serializer):
"""Serializer for AI translate requests."""
language = serializers.ChoiceField(
choices=tuple(enums.ALL_LANGUAGES.items()), required=True
)
text = serializers.CharField(required=True)
def validate_text(self, value):
"""Ensure the text field is not empty."""
if len(value.strip()) == 0:
raise serializers.ValidationError("Text field cannot be empty.")
return value

View File

@@ -1,8 +1,14 @@
"""Util to generate S3 authorization headers for object storage access control"""
import time
from abc import ABC, abstractmethod
from django.conf import settings
from django.core.cache import cache
from django.core.files.storage import default_storage
import botocore
from rest_framework.throttling import BaseThrottle
def generate_s3_authorization_headers(key):
@@ -31,3 +37,93 @@ def generate_s3_authorization_headers(key):
auth.add_auth(request)
return request
class AIBaseRateThrottle(BaseThrottle, ABC):
"""Base throttle class for AI-related rate limiting with backoff."""
def __init__(self, rates):
"""Initialize instance attributes with configurable rates."""
super().__init__()
self.rates = rates
self.cache_key = None
self.recent_requests_minute = 0
self.recent_requests_hour = 0
self.recent_requests_day = 0
@abstractmethod
def get_cache_key(self, request, view):
"""Abstract method to generate cache key for throttling."""
def allow_request(self, request, view):
"""Check if the request is allowed based on rate limits."""
self.cache_key = self.get_cache_key(request, view)
if not self.cache_key:
return True # Allow if no cache key is generated
now = time.time()
history = cache.get(self.cache_key, [])
# Keep requests within the last 24 hours
history = [req for req in history if req > now - 86400]
# Calculate recent requests
self.recent_requests_minute = len([req for req in history if req > now - 60])
self.recent_requests_hour = len([req for req in history if req > now - 3600])
self.recent_requests_day = len(history)
# Check rate limits
if self.recent_requests_minute >= self.rates["minute"]:
return False
if self.recent_requests_hour >= self.rates["hour"]:
return False
if self.recent_requests_day >= self.rates["day"]:
return False
# Log the request
history.append(now)
cache.set(self.cache_key, history, timeout=86400)
return True
def wait(self):
"""Implement a backoff strategy by increasing wait time based on limits hit."""
if self.recent_requests_day >= self.rates["day"]:
return 86400
if self.recent_requests_hour >= self.rates["hour"]:
return 3600
if self.recent_requests_minute >= self.rates["minute"]:
return 60
return None
class AIDocumentRateThrottle(AIBaseRateThrottle):
"""Throttle for limiting AI requests per document with backoff."""
def __init__(self, *args, **kwargs):
super().__init__(settings.AI_DOCUMENT_RATE_THROTTLE_RATES)
def get_cache_key(self, request, view):
"""Include document ID in the cache key."""
document_id = view.kwargs["pk"]
return f"document_{document_id}_throttle_ai"
class AIUserRateThrottle(AIBaseRateThrottle):
"""Throttle that limits requests per user or IP with backoff and rate limits."""
def __init__(self, *args, **kwargs):
super().__init__(settings.AI_USER_RATE_THROTTLE_RATES)
def get_cache_key(self, request, view=None):
"""Generate a cache key based on the user ID or IP for anonymous users."""
if request.user.is_authenticated:
return f"user_{request.user.id!s}_throttle_ai"
return f"anonymous_{self.get_ident(request)}_throttle_ai"
def get_ident(self, request):
"""Return the request IP address."""
x_forwarded_for = request.META.get("HTTP_X_FORWARDED_FOR")
return (
x_forwarded_for.split(",")[0]
if x_forwarded_for
else request.META.get("REMOTE_ADDR")
)

View File

@@ -1,6 +1,5 @@
"""API endpoints"""
import os
import re
import uuid
from urllib.parse import urlparse
@@ -22,6 +21,7 @@ from rest_framework import (
decorators,
exceptions,
filters,
metadata,
mixins,
pagination,
status,
@@ -31,7 +31,8 @@ from rest_framework import (
response as drf_response,
)
from core import models
from core import enums, models
from core.services.ai_services import AIService
from . import permissions, serializers, utils
@@ -303,6 +304,23 @@ class ResourceAccessViewsetMixin:
serializer.save()
class DocumentMetadata(metadata.SimpleMetadata):
"""Custom metadata class to add information"""
def determine_metadata(self, request, view):
"""Add language choices only for the list endpoint."""
simple_metadata = super().determine_metadata(request, view)
if request.path.endswith("/documents/"):
simple_metadata["actions"]["POST"]["language"] = {
"choices": [
{"value": code, "display_name": name}
for code, name in enums.ALL_LANGUAGES.items()
]
}
return simple_metadata
class DocumentViewSet(
ResourceViewsetMixin,
mixins.CreateModelMixin,
@@ -320,6 +338,7 @@ class DocumentViewSet(
resource_field_name = "document"
queryset = models.Document.objects.all()
ordering = ["-updated_at"]
metadata_class = DocumentMetadata
def list(self, request, *args, **kwargs):
"""Restrict resources returned by the list endpoint"""
@@ -456,10 +475,7 @@ class DocumentViewSet(
serializer = serializers.LinkDocumentSerializer(
document, data=request.data, partial=True
)
if not serializer.is_valid():
return drf_response.Response(
serializer.errors, status=status.HTTP_400_BAD_REQUEST
)
serializer.is_valid(raise_exception=True)
serializer.save()
return drf_response.Response(serializer.data, status=status.HTTP_200_OK)
@@ -472,19 +488,23 @@ class DocumentViewSet(
# Validate metadata in payload
serializer = serializers.FileUploadSerializer(data=request.data)
if not serializer.is_valid():
return drf_response.Response(
serializer.errors, status=status.HTTP_400_BAD_REQUEST
)
# Extract the file extension from the original filename
file = serializer.validated_data["file"]
extension = os.path.splitext(file.name)[1]
serializer.is_valid(raise_exception=True)
# Generate a generic yet unique filename to store the image in object storage
file_id = uuid.uuid4()
key = f"{document.key_base}/{ATTACHMENTS_FOLDER:s}/{file_id!s}{extension:s}"
extension = serializer.validated_data["expected_extension"]
key = f"{document.key_base}/{ATTACHMENTS_FOLDER:s}/{file_id!s}.{extension:s}"
# Prepare metadata for storage
extra_args = {"Metadata": {"owner": str(request.user.id)}}
if serializer.validated_data["is_unsafe"]:
extra_args["Metadata"]["is_unsafe"] = "true"
file = serializer.validated_data["file"]
default_storage.connection.meta.client.upload_fileobj(
file, default_storage.bucket_name, key, ExtraArgs=extra_args
)
default_storage.save(key, file)
return drf_response.Response(
{"file": f"{settings.MEDIA_URL:s}{key:s}"}, status=status.HTTP_201_CREATED
)
@@ -531,6 +551,63 @@ class DocumentViewSet(
request = utils.generate_s3_authorization_headers(f"{pk:s}/{attachment_key:s}")
return drf_response.Response("authorized", headers=request.headers, status=200)
@decorators.action(
detail=True,
methods=["post"],
name="Apply a transformation action on a piece of text with AI",
url_path="ai-transform",
throttle_classes=[utils.AIDocumentRateThrottle, utils.AIUserRateThrottle],
)
def ai_transform(self, request, *args, **kwargs):
"""
POST /api/v1.0/documents/<resource_id>/ai-transform
with expected data:
- text: str
- action: str [prompt, correct, rephrase, summarize]
Return JSON response with the processed text.
"""
# Check permissions first
self.get_object()
serializer = serializers.AITransformSerializer(data=request.data)
serializer.is_valid(raise_exception=True)
text = serializer.validated_data["text"]
action = serializer.validated_data["action"]
response = AIService().transform(text, action)
return drf_response.Response(response, status=status.HTTP_200_OK)
@decorators.action(
detail=True,
methods=["post"],
name="Translate a piece of text with AI",
serializer_class=serializers.AITranslateSerializer,
url_path="ai-translate",
throttle_classes=[utils.AIDocumentRateThrottle, utils.AIUserRateThrottle],
)
def ai_translate(self, request, *args, **kwargs):
"""
POST /api/v1.0/documents/<resource_id>/ai-translate
with expected data:
- text: str
- language: str [settings.LANGUAGES]
Return JSON response with the translated text.
"""
# Check permissions first
self.get_object()
serializer = self.get_serializer(data=request.data)
serializer.is_valid(raise_exception=True)
text = serializer.validated_data["text"]
language = serializer.validated_data["language"]
response = AIService().translate(text, language)
return drf_response.Response(response, status=status.HTTP_200_OK)
class DocumentAccessViewSet(
ResourceAccessViewsetMixin,
@@ -576,8 +653,12 @@ class DocumentAccessViewSet(
"""Add a new access to the document and send an email to the new added user."""
access = serializer.save()
language = self.request.headers.get("Content-Language", "en-us")
access.document.email_invitation(
language, access.user.email, access.role, self.request.user.email
language,
access.user.email,
access.role,
self.request.user,
)
@@ -778,6 +859,7 @@ class InvitationViewset(
invitation = serializer.save()
language = self.request.headers.get("Content-Language", "en-us")
invitation.document.email_invitation(
language, invitation.email, invitation.role, self.request.user.email
language, invitation.email, invitation.role, self.request.user
)

View File

@@ -2,15 +2,11 @@
Core application enums declaration
"""
from django.conf import global_settings, settings
from django.conf import global_settings
from django.utils.translation import gettext_lazy as _
# Django sets `LANGUAGES` by default with all supported languages. We can use it for
# the choice of languages which should not be limited to the few languages active in
# the app.
# In Django's code base, `LANGUAGES` is set by default with all supported languages.
# We can use it for the choice of languages which should not be limited to the few languages
# active in the app.
# pylint: disable=no-member
ALL_LANGUAGES = getattr(
settings,
"ALL_LANGUAGES",
[(language, _(name)) for language, name in global_settings.LANGUAGES],
)
ALL_LANGUAGES = {language: _(name) for language, name in global_settings.LANGUAGES}

View File

@@ -27,6 +27,24 @@ class UserFactory(factory.django.DjangoModelFactory):
language = factory.fuzzy.FuzzyChoice([lang[0] for lang in settings.LANGUAGES])
password = make_password("password")
@factory.post_generation
def with_owned_document(self, create, extracted, **kwargs):
"""
Create a document for which the user is owner to check
that there is no interference
"""
if create and (extracted is True):
UserDocumentAccessFactory(user=self, role="owner")
@factory.post_generation
def with_owned_template(self, create, extracted, **kwargs):
"""
Create a template for which the user is owner to check
that there is no interference
"""
if create and (extracted is True):
UserTemplateAccessFactory(user=self, role="owner")
class DocumentFactory(factory.django.DjangoModelFactory):
"""A factory to create documents"""

View File

@@ -508,6 +508,8 @@ class Document(BaseModel):
can_get = bool(roles)
return {
"ai_transform": is_owner_or_admin or is_editor,
"ai_translate": is_owner_or_admin or is_editor,
"attachment_upload": is_owner_or_admin or is_editor,
"destroy": RoleChoices.OWNER in roles,
"link_configuration": is_owner_or_admin,
@@ -520,15 +522,18 @@ class Document(BaseModel):
"versions_retrieve": can_get_versions,
}
def email_invitation(self, language, email, role, username_sender):
def email_invitation(self, language, email, role, sender):
"""Send email invitation."""
sender_name = sender.full_name or sender.email
domain = Site.objects.get_current().domain
try:
with override(language):
title = _("%(username)s shared a document with you: %(document)s") % {
"username": username_sender,
title = _(
"%(sender_name)s shared a document with you: %(document)s"
) % {
"sender_name": sender_name,
"document": self.title,
}
template_vars = {
@@ -536,7 +541,10 @@ class Document(BaseModel):
"domain": domain,
"document": self,
"link": f"{domain}/docs/{self.id}/",
"username": username_sender,
"sender_name": sender_name,
"sender_name_email": f"{sender.full_name} ({sender.email})"
if sender.full_name
else sender.email,
"role": RoleChoices(role).label.lower(),
}
msg_html = render_to_string("mail/html/invitation.html", template_vars)

View File

View File

@@ -0,0 +1,89 @@
"""AI services."""
import json
import re
from django.conf import settings
from django.core.exceptions import ImproperlyConfigured
from openai import OpenAI
from core import enums
AI_ACTIONS = {
"prompt": (
"Answer the prompt in markdown format. Return JSON: "
'{"answer": "Your markdown answer"}. '
"Do not provide any other information."
),
"correct": (
"Correct grammar and spelling of the markdown text, "
"preserving language and markdown formatting. "
'Return JSON: {"answer": "your corrected markdown text"}. '
"Do not provide any other information."
),
"rephrase": (
"Rephrase the given markdown text, "
"preserving language and markdown formatting. "
'Return JSON: {"answer": "your rephrased markdown text"}. '
"Do not provide any other information."
),
"summarize": (
"Summarize the markdown text, preserving language and markdown formatting. "
'Return JSON: {"answer": "your markdown summary"}. '
"Do not provide any other information."
),
}
AI_TRANSLATE = (
"Translate the markdown text to {language:s}, preserving markdown formatting. "
'Return JSON: {{"answer": "your translated markdown text in {language:s}"}}. '
"Do not provide any other information."
)
class AIService:
"""Service class for AI-related operations."""
def __init__(self):
"""Ensure that the AI configuration is set properly."""
if (
settings.AI_BASE_URL is None
or settings.AI_API_KEY is None
or settings.AI_MODEL is None
):
raise ImproperlyConfigured("AI configuration not set")
self.client = OpenAI(base_url=settings.AI_BASE_URL, api_key=settings.AI_API_KEY)
def call_ai_api(self, system_content, text):
"""Helper method to call the OpenAI API and process the response."""
response = self.client.chat.completions.create(
model=settings.AI_MODEL,
response_format={"type": "json_object"},
messages=[
{"role": "system", "content": system_content},
{"role": "user", "content": json.dumps({"markdown_input": text})},
],
)
content = response.choices[0].message.content
sanitized_content = re.sub(r"(?<!\\)\n", "\\\\n", content)
sanitized_content = re.sub(r"(?<!\\)\t", "\\\\t", sanitized_content)
json_response = json.loads(sanitized_content)
if "answer" not in json_response:
raise RuntimeError("AI response does not contain an answer")
return json_response
def transform(self, text, action):
"""Transform text based on specified action."""
system_content = AI_ACTIONS[action]
return self.call_ai_api(system_content, text)
def translate(self, text, language):
"""Translate text to a specified language."""
language_display = enums.ALL_LANGUAGES.get(language, language)
system_content = AI_TRANSLATE.format(language=language_display)
return self.call_ai_api(system_content, text)

View File

@@ -149,7 +149,7 @@ def test_api_document_accesses_retrieve_authenticated_unrelated():
Authenticated users should not be allowed to retrieve a document access for
a document to which they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -246,7 +246,7 @@ def test_api_document_accesses_update_authenticated_unrelated():
Authenticated users should not be allowed to update a document access for a document to which
they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -279,7 +279,7 @@ def test_api_document_accesses_update_authenticated_reader_or_editor(
via, role, mock_user_teams
):
"""Readers or editors of a document should not be allowed to update its accesses."""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -321,7 +321,7 @@ def test_api_document_accesses_update_administrator_except_owner(via, mock_user_
A user who is a direct administrator in a document should be allowed to update a user
access for this document, as long as they don't try to set the role to owner.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -378,7 +378,7 @@ def test_api_document_accesses_update_administrator_from_owner(via, mock_user_te
A user who is an administrator in a document, should not be allowed to update
the user access of an "owner" for this document.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -425,7 +425,7 @@ def test_api_document_accesses_update_administrator_to_owner(via, mock_user_team
A user who is an administrator in a document, should not be allowed to update
the user access of another user to grant document ownership.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -479,7 +479,7 @@ def test_api_document_accesses_update_owner(via, mock_user_teams):
A user who is an owner in a document should be allowed to update
a user access for this document whatever the role.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -535,7 +535,7 @@ def test_api_document_accesses_update_owner_self(via, mock_user_teams):
A user who is owner of a document should be allowed to update
their own user access provided there are other owners in the document.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -605,7 +605,7 @@ def test_api_document_accesses_delete_authenticated():
Authenticated users should not be allowed to delete a document access for a
document to which they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -617,7 +617,7 @@ def test_api_document_accesses_delete_authenticated():
)
assert response.status_code == 403
assert models.DocumentAccess.objects.count() == 1
assert models.DocumentAccess.objects.count() == 2
@pytest.mark.parametrize("role", ["reader", "editor"])
@@ -627,7 +627,7 @@ def test_api_document_accesses_delete_reader_or_editor(via, role, mock_user_team
Authenticated users should not be allowed to delete a document access for a
document in which they are a simple reader or editor.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -643,7 +643,7 @@ def test_api_document_accesses_delete_reader_or_editor(via, role, mock_user_team
access = factories.UserDocumentAccessFactory(document=document)
assert models.DocumentAccess.objects.count() == 2
assert models.DocumentAccess.objects.count() == 3
assert models.DocumentAccess.objects.filter(user=access.user).exists()
response = client.delete(
@@ -651,7 +651,7 @@ def test_api_document_accesses_delete_reader_or_editor(via, role, mock_user_team
)
assert response.status_code == 403
assert models.DocumentAccess.objects.count() == 2
assert models.DocumentAccess.objects.count() == 3
@pytest.mark.parametrize("via", VIA)
@@ -699,7 +699,7 @@ def test_api_document_accesses_delete_administrator_on_owners(via, mock_user_tea
Users who are administrators in a document should not be allowed to delete an ownership
access from the document.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -717,7 +717,7 @@ def test_api_document_accesses_delete_administrator_on_owners(via, mock_user_tea
access = factories.UserDocumentAccessFactory(document=document, role="owner")
assert models.DocumentAccess.objects.count() == 2
assert models.DocumentAccess.objects.count() == 3
assert models.DocumentAccess.objects.filter(user=access.user).exists()
response = client.delete(
@@ -725,7 +725,7 @@ def test_api_document_accesses_delete_administrator_on_owners(via, mock_user_tea
)
assert response.status_code == 403
assert models.DocumentAccess.objects.count() == 2
assert models.DocumentAccess.objects.count() == 3
@pytest.mark.parametrize("via", VIA)
@@ -766,7 +766,7 @@ def test_api_document_accesses_delete_owners_last_owner(via, mock_user_teams):
"""
It should not be possible to delete the last owner access from a document
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -783,10 +783,10 @@ def test_api_document_accesses_delete_owners_last_owner(via, mock_user_teams):
document=document, team="lasuite", role="owner"
)
assert models.DocumentAccess.objects.count() == 1
assert models.DocumentAccess.objects.count() == 2
response = client.delete(
f"/api/v1.0/documents/{document.id!s}/accesses/{access.id!s}/",
)
assert response.status_code == 403
assert models.DocumentAccess.objects.count() == 1
assert models.DocumentAccess.objects.count() == 2

View File

@@ -18,13 +18,13 @@ pytestmark = pytest.mark.django_db
def test_api_document_accesses_create_anonymous():
"""Anonymous users should not be allowed to create document accesses."""
user = factories.UserFactory()
document = factories.DocumentFactory()
other_user = factories.UserFactory()
response = APIClient().post(
f"/api/v1.0/documents/{document.id!s}/accesses/",
{
"user": str(user.id),
"user_id": str(other_user.id),
"document": str(document.id),
"role": random.choice(models.RoleChoices.choices)[0],
},
@@ -43,7 +43,7 @@ def test_api_document_accesses_create_authenticated_unrelated():
Authenticated users should not be allowed to create document accesses for a document to
which they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -54,7 +54,7 @@ def test_api_document_accesses_create_authenticated_unrelated():
response = client.post(
f"/api/v1.0/documents/{document.id!s}/accesses/",
{
"user": str(other_user.id),
"user_id": str(other_user.id),
},
format="json",
)
@@ -69,7 +69,7 @@ def test_api_document_accesses_create_authenticated_reader_or_editor(
via, role, mock_user_teams
):
"""Readers or editors of a document should not be allowed to create document accesses."""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -89,7 +89,7 @@ def test_api_document_accesses_create_authenticated_reader_or_editor(
response = client.post(
f"/api/v1.0/documents/{document.id!s}/accesses/",
{
"user": str(other_user.id),
"user_id": str(other_user.id),
"role": new_role,
},
format="json",
@@ -107,7 +107,7 @@ def test_api_document_accesses_create_authenticated_administrator(via, mock_user
except for the "owner" role.
An email should be sent to the accesses to notify them of the adding.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -129,7 +129,7 @@ def test_api_document_accesses_create_authenticated_administrator(via, mock_user
response = client.post(
f"/api/v1.0/documents/{document.id!s}/accesses/",
{
"user": str(other_user.id),
"user_id": str(other_user.id),
"role": "owner",
},
format="json",
@@ -171,7 +171,10 @@ def test_api_document_accesses_create_authenticated_administrator(via, mock_user
email = mail.outbox[0]
assert email.to == [other_user["email"]]
email_content = " ".join(email.body.split())
assert f"{user.email} shared a document with you: {document.title}" in email_content
assert (
f"{user.full_name} shared a document with you: {document.title}"
in email_content
)
assert "docs/" + str(document.id) + "/" in email_content
@@ -225,5 +228,8 @@ def test_api_document_accesses_create_authenticated_owner(via, mock_user_teams):
email = mail.outbox[0]
assert email.to == [other_user["email"]]
email_content = " ".join(email.body.split())
assert f"{user.email} shared a document with you: {document.title}" in email_content
assert (
f"{user.full_name} shared a document with you: {document.title}"
in email_content
)
assert "docs/" + str(document.id) + "/" in email_content

View File

@@ -119,7 +119,7 @@ def test_api_document_invitations__create__privileged_members(
assert email.to == ["guest@example.com"]
email_content = " ".join(email.body.split())
assert (
f"{user.email} shared a document with you: {document.title}"
f"{user.full_name} shared a document with you: {document.title}"
in email_content
)
else:
@@ -162,7 +162,7 @@ def test_api_document_invitations__create__email_from_content_language():
email_content = " ".join(email.body.split())
assert (
f"{user.email} a partagé un document avec vous: {document.title}"
f"{user.full_name} a partagé un document avec vous: {document.title}"
in email_content
)
@@ -201,8 +201,52 @@ def test_api_document_invitations__create__email_from_content_language_not_suppo
assert email.to == ["guest@example.com"]
email_content = " ".join(email.body.split())
assert (
f"{user.full_name} shared a document with you: {document.title}"
in email_content
)
def test_api_document_invitations__create__email__full_name_empty():
"""
If the full name of the user is empty, it will display the email address.
"""
user = factories.UserFactory(full_name="")
document = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=document, user=user, role="owner")
invitation_values = {
"email": "guest@example.com",
"role": "reader",
}
assert len(mail.outbox) == 0
client = APIClient()
client.force_login(user)
response = client.post(
f"/api/v1.0/documents/{document.id}/invitations/",
invitation_values,
format="json",
headers={"Content-Language": "not-supported"},
)
assert response.status_code == status.HTTP_201_CREATED
assert response.json()["email"] == "guest@example.com"
assert models.Invitation.objects.count() == 1
assert len(mail.outbox) == 1
email = mail.outbox[0]
assert email.to == ["guest@example.com"]
email_content = " ".join(email.body.split())
assert f"{user.email} shared a document with you: {document.title}" in email_content
assert (
f'{user.email} invited you with the role "reader" on the '
f"following document : {document.title}" in email_content
)
def test_api_document_invitations__create__issuer_and_document_override():
@@ -557,7 +601,7 @@ def test_api_document_invitations__update__forbidden__not_authenticated(
"""
Update of invitations is currently forbidden.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
invitation = factories.InvitationFactory()
if via == USER:
factories.UserDocumentAccessFactory(
@@ -597,7 +641,7 @@ def test_api_document_invitations__delete__anonymous():
def test_api_document_invitations__delete__authenticated_outsider():
"""Members unrelated to a document should not be allowed to cancel invitations."""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
document = factories.DocumentFactory()
invitation = factories.InvitationFactory(document=document)
@@ -640,7 +684,7 @@ def test_api_document_invitations__delete__privileged_members(
@pytest.mark.parametrize("via", VIA)
def test_api_document_invitations_delete_readers_or_editors(via, role, mock_user_teams):
"""Readers or editors should not be able to cancel invitation."""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
document = factories.DocumentFactory()
if via == USER:
factories.UserDocumentAccessFactory(document=document, user=user, role=role)

View File

@@ -39,7 +39,7 @@ def test_api_document_versions_list_authenticated_unrelated(reach):
Authenticated users should not be allowed to list document versions for a document
to which they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -231,7 +231,7 @@ def test_api_document_versions_retrieve_authenticated_unrelated(reach):
Authenticated users should not be allowed to retrieve specific versions for a
document to which they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -475,7 +475,7 @@ def test_api_document_versions_delete_authenticated(reach):
Authenticated users should not be allowed to delete a document version for a
public document to which they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -500,7 +500,7 @@ def test_api_document_versions_delete_reader_or_editor(via, role, mock_user_team
Authenticated users should not be allowed to delete a document version for a
document in which they are a simple reader or editor.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)

View File

@@ -0,0 +1,346 @@
"""
Test AI transform API endpoint for users in impress's core app.
"""
from unittest.mock import MagicMock, patch
from django.core.cache import cache
from django.test import override_settings
import pytest
from rest_framework.test import APIClient
from core import factories
from core.tests.conftest import TEAM, USER, VIA
pytestmark = pytest.mark.django_db
@pytest.fixture(autouse=True)
def clear_cache():
"""Fixture to clear the cache before each test."""
cache.clear()
@pytest.fixture
def ai_settings():
"""Fixture to set AI settings."""
with override_settings(
AI_BASE_URL="http://example.com", AI_API_KEY="test-key", AI_MODEL="llama"
):
yield
@pytest.mark.parametrize(
"reach, role",
[
("restricted", "reader"),
("restricted", "editor"),
("authenticated", "reader"),
("authenticated", "editor"),
("public", "reader"),
],
)
def test_api_documents_ai_transform_anonymous_forbidden(reach, role):
"""
Anonymous users should not be able to request AI transform if the link reach
and role don't allow it.
"""
document = factories.DocumentFactory(link_reach=reach, link_role=role)
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
response = APIClient().post(url, {"text": "hello", "action": "prompt"})
assert response.status_code == 401
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_transform_anonymous_success(mock_create):
"""
Anonymous users should be able to request AI transform to a document
if the link reach and role permit it.
"""
document = factories.DocumentFactory(link_reach="public", link_role="editor")
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
response = APIClient().post(url, {"text": "Hello", "action": "summarize"})
assert response.status_code == 200
assert response.json() == {"answer": "Salut"}
mock_create.assert_called_once_with(
model="llama",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": (
"Summarize the markdown text, preserving language and markdown formatting. "
'Return JSON: {"answer": "your markdown summary"}. Do not provide any other '
"information."
),
},
{"role": "user", "content": '{"markdown_input": "Hello"}'},
],
)
@pytest.mark.parametrize(
"reach, role",
[
("restricted", "reader"),
("restricted", "editor"),
("authenticated", "reader"),
("public", "reader"),
],
)
def test_api_documents_ai_transform_authenticated_forbidden(reach, role):
"""
Users who are not related to a document can't request AI transform if the
link reach and role don't allow it.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach=reach, link_role=role)
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
response = client.post(url, {"text": "Hello", "action": "prompt"})
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
@pytest.mark.parametrize(
"reach, role",
[
("authenticated", "editor"),
("public", "editor"),
],
)
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_transform_authenticated_success(mock_create, reach, role):
"""
Autenticated who are not related to a document should be able to request AI transform
if the link reach and role permit it.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach=reach, link_role=role)
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
response = client.post(url, {"text": "Hello", "action": "prompt"})
assert response.status_code == 200
assert response.json() == {"answer": "Salut"}
mock_create.assert_called_once_with(
model="llama",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": (
'Answer the prompt in markdown format. Return JSON: {"answer": '
'"Your markdown answer"}. Do not provide any other information.'
),
},
{"role": "user", "content": '{"markdown_input": "Hello"}'},
],
)
@pytest.mark.parametrize("via", VIA)
def test_api_documents_ai_transform_reader(via, mock_user_teams):
"""
Users who are simple readers on a document should not be allowed to request AI transform.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_role="reader")
if via == USER:
factories.UserDocumentAccessFactory(document=document, user=user, role="reader")
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamDocumentAccessFactory(
document=document, team="lasuite", role="reader"
)
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
response = client.post(url, {"text": "Hello", "action": "prompt"})
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
@pytest.mark.parametrize("role", ["editor", "administrator", "owner"])
@pytest.mark.parametrize("via", VIA)
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_transform_success(mock_create, via, role, mock_user_teams):
"""
Editors, administrators and owners of a document should be able to request AI transform.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory()
if via == USER:
factories.UserDocumentAccessFactory(document=document, user=user, role=role)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamDocumentAccessFactory(
document=document, team="lasuite", role=role
)
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
response = client.post(url, {"text": "Hello", "action": "prompt"})
assert response.status_code == 200
assert response.json() == {"answer": "Salut"}
mock_create.assert_called_once_with(
model="llama",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": (
'Answer the prompt in markdown format. Return JSON: {"answer": '
'"Your markdown answer"}. Do not provide any other information.'
),
},
{"role": "user", "content": '{"markdown_input": "Hello"}'},
],
)
def test_api_documents_ai_transform_empty_text():
"""The text should not be empty when requesting AI transform."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="public", link_role="editor")
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
response = client.post(url, {"text": " ", "action": "prompt"})
assert response.status_code == 400
assert response.json() == {"text": ["This field may not be blank."]}
def test_api_documents_ai_transform_invalid_action():
"""The action should valid when requesting AI transform."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="public", link_role="editor")
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
response = client.post(url, {"text": "Hello", "action": "invalid"})
assert response.status_code == 400
assert response.json() == {"action": ['"invalid" is not a valid choice.']}
@override_settings(AI_DOCUMENT_RATE_THROTTLE_RATES={"minute": 3, "hour": 6, "day": 10})
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_transform_throttling_document(mock_create):
"""
Throttling per document should be triggered on the AI transform endpoint.
For full throttle class test see: `test_api_utils_ai_document_rate_throttles`
"""
client = APIClient()
document = factories.DocumentFactory(link_reach="public", link_role="editor")
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
for _ in range(3):
user = factories.UserFactory()
client.force_login(user)
response = client.post(url, {"text": "Hello", "action": "summarize"})
assert response.status_code == 200
assert response.json() == {"answer": "Salut"}
user = factories.UserFactory()
client.force_login(user)
response = client.post(url, {"text": "Hello", "action": "summarize"})
assert response.status_code == 429
assert response.json() == {
"detail": "Request was throttled. Expected available in 60 seconds."
}
@override_settings(AI_USER_RATE_THROTTLE_RATES={"minute": 3, "hour": 6, "day": 10})
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_transform_throttling_user(mock_create):
"""
Throttling per user should be triggered on the AI transform endpoint.
For full throttle class test see: `test_api_utils_ai_user_rate_throttles`
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
)
for _ in range(3):
document = factories.DocumentFactory(link_reach="public", link_role="editor")
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
response = client.post(url, {"text": "Hello", "action": "summarize"})
assert response.status_code == 200
assert response.json() == {"answer": "Salut"}
document = factories.DocumentFactory(link_reach="public", link_role="editor")
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
response = client.post(url, {"text": "Hello", "action": "summarize"})
assert response.status_code == 429
assert response.json() == {
"detail": "Request was throttled. Expected available in 60 seconds."
}

View File

@@ -0,0 +1,370 @@
"""
Test AI translate API endpoint for users in impress's core app.
"""
from unittest.mock import MagicMock, patch
from django.core.cache import cache
from django.test import override_settings
import pytest
from rest_framework.test import APIClient
from core import factories
from core.tests.conftest import TEAM, USER, VIA
pytestmark = pytest.mark.django_db
@pytest.fixture(autouse=True)
def clear_cache():
"""Fixture to clear the cache before each test."""
cache.clear()
@pytest.fixture
def ai_settings():
"""Fixture to set AI settings."""
with override_settings(
AI_BASE_URL="http://example.com", AI_API_KEY="test-key", AI_MODEL="llama"
):
yield
def test_api_documents_ai_translate_viewset_options_metadata():
"""The documents endpoint should give us the list of available languages."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory(link_reach="public", link_role="editor")
response = APIClient().options("/api/v1.0/documents/")
assert response.status_code == 200
metadata = response.json()
assert metadata["name"] == "Document List"
assert metadata["actions"]["POST"]["language"]["choices"][0] == {
"value": "af",
"display_name": "Afrikaans",
}
@pytest.mark.parametrize(
"reach, role",
[
("restricted", "reader"),
("restricted", "editor"),
("authenticated", "reader"),
("authenticated", "editor"),
("public", "reader"),
],
)
def test_api_documents_ai_translate_anonymous_forbidden(reach, role):
"""
Anonymous users should not be able to request AI translate if the link reach
and role don't allow it.
"""
document = factories.DocumentFactory(link_reach=reach, link_role=role)
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
response = APIClient().post(url, {"text": "hello", "language": "es"})
assert response.status_code == 401
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_translate_anonymous_success(mock_create):
"""
Anonymous users should be able to request AI translate to a document
if the link reach and role permit it.
"""
document = factories.DocumentFactory(link_reach="public", link_role="editor")
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
response = APIClient().post(url, {"text": "Hello", "language": "es"})
assert response.status_code == 200
assert response.json() == {"answer": "Salut"}
mock_create.assert_called_once_with(
model="llama",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": (
"Translate the markdown text to Spanish, preserving markdown formatting. "
'Return JSON: {"answer": "your translated markdown text in Spanish"}. '
"Do not provide any other information."
),
},
{"role": "user", "content": '{"markdown_input": "Hello"}'},
],
)
@pytest.mark.parametrize(
"reach, role",
[
("restricted", "reader"),
("restricted", "editor"),
("authenticated", "reader"),
("public", "reader"),
],
)
def test_api_documents_ai_translate_authenticated_forbidden(reach, role):
"""
Users who are not related to a document can't request AI translate if the
link reach and role don't allow it.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach=reach, link_role=role)
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
response = client.post(url, {"text": "Hello", "language": "es"})
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
@pytest.mark.parametrize(
"reach, role",
[
("authenticated", "editor"),
("public", "editor"),
],
)
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_translate_authenticated_success(mock_create, reach, role):
"""
Autenticated who are not related to a document should be able to request AI translate
if the link reach and role permit it.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach=reach, link_role=role)
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
response = client.post(url, {"text": "Hello", "language": "es-co"})
assert response.status_code == 200
assert response.json() == {"answer": "Salut"}
mock_create.assert_called_once_with(
model="llama",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": (
"Translate the markdown text to Colombian Spanish, "
"preserving markdown formatting. Return JSON: "
'{"answer": "your translated markdown text in Colombian Spanish"}. '
"Do not provide any other information."
),
},
{"role": "user", "content": '{"markdown_input": "Hello"}'},
],
)
@pytest.mark.parametrize("via", VIA)
def test_api_documents_ai_translate_reader(via, mock_user_teams):
"""
Users who are simple readers on a document should not be allowed to request AI translate.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_role="reader")
if via == USER:
factories.UserDocumentAccessFactory(document=document, user=user, role="reader")
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamDocumentAccessFactory(
document=document, team="lasuite", role="reader"
)
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
response = client.post(url, {"text": "Hello", "language": "es"})
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
@pytest.mark.parametrize("role", ["editor", "administrator", "owner"])
@pytest.mark.parametrize("via", VIA)
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_translate_success(mock_create, via, role, mock_user_teams):
"""
Editors, administrators and owners of a document should be able to request AI translate.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory()
if via == USER:
factories.UserDocumentAccessFactory(document=document, user=user, role=role)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamDocumentAccessFactory(
document=document, team="lasuite", role=role
)
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
response = client.post(url, {"text": "Hello", "language": "es-co"})
assert response.status_code == 200
assert response.json() == {"answer": "Salut"}
mock_create.assert_called_once_with(
model="llama",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": (
"Translate the markdown text to Colombian Spanish, "
"preserving markdown formatting. Return JSON: "
'{"answer": "your translated markdown text in Colombian Spanish"}. '
"Do not provide any other information."
),
},
{"role": "user", "content": '{"markdown_input": "Hello"}'},
],
)
def test_api_documents_ai_translate_empty_text():
"""The text should not be empty when requesting AI translate."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="public", link_role="editor")
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
response = client.post(url, {"text": " ", "language": "es"})
assert response.status_code == 400
assert response.json() == {"text": ["This field may not be blank."]}
def test_api_documents_ai_translate_invalid_action():
"""The action should valid when requesting AI translate."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="public", link_role="editor")
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
response = client.post(url, {"text": "Hello", "language": "invalid"})
assert response.status_code == 400
assert response.json() == {"language": ['"invalid" is not a valid choice.']}
@override_settings(AI_DOCUMENT_RATE_THROTTLE_RATES={"minute": 3, "hour": 6, "day": 10})
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_translate_throttling_document(mock_create):
"""
Throttling per document should be triggered on the AI translate endpoint.
For full throttle class test see: `test_api_utils_ai_document_rate_throttles`
"""
client = APIClient()
document = factories.DocumentFactory(link_reach="public", link_role="editor")
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
for _ in range(3):
user = factories.UserFactory()
client.force_login(user)
response = client.post(url, {"text": "Hello", "language": "es"})
assert response.status_code == 200
assert response.json() == {"answer": "Salut"}
user = factories.UserFactory()
client.force_login(user)
response = client.post(url, {"text": "Hello", "language": "es"})
assert response.status_code == 429
assert response.json() == {
"detail": "Request was throttled. Expected available in 60 seconds."
}
@override_settings(AI_USER_RATE_THROTTLE_RATES={"minute": 3, "hour": 6, "day": 10})
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_translate_throttling_user(mock_create):
"""
Throttling per user should be triggered on the AI translate endpoint.
For full throttle class test see: `test_api_utils_ai_user_rate_throttles`
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
)
for _ in range(3):
document = factories.DocumentFactory(link_reach="public", link_role="editor")
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
response = client.post(url, {"text": "Hello", "language": "es"})
assert response.status_code == 200
assert response.json() == {"answer": "Salut"}
document = factories.DocumentFactory(link_reach="public", link_role="editor")
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
response = client.post(url, {"text": "Hello", "language": "es"})
assert response.status_code == 429
assert response.json() == {
"detail": "Request was throttled. Expected available in 60 seconds."
}

View File

@@ -5,7 +5,7 @@ Test file uploads API endpoint for users in impress's core app.
import re
import uuid
from django.core.files.base import ContentFile
from django.core.files.storage import default_storage
from django.core.files.uploadedfile import SimpleUploadedFile
import pytest
@@ -16,6 +16,12 @@ from core.tests.conftest import TEAM, USER, VIA
pytestmark = pytest.mark.django_db
PIXEL = (
b"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x01\x00\x00\x00\x01\x08\x06\x00"
b"\x00\x00\x1f\x15\xc4\x89\x00\x00\x00\nIDATx\x9cc\xf8\xff\xff?\x00\x05\xfe\x02\xfe"
b"\xa7V\xbd\xfa\x00\x00\x00\x00IEND\xaeB`\x82"
)
@pytest.mark.parametrize(
"reach, role",
@@ -33,7 +39,7 @@ def test_api_documents_attachment_upload_anonymous_forbidden(reach, role):
and role don't allow it.
"""
document = factories.DocumentFactory(link_reach=reach, link_role=role)
file = SimpleUploadedFile("test_file.jpg", b"Dummy content")
file = SimpleUploadedFile(name="test.png", content=PIXEL, content_type="image/png")
url = f"/api/v1.0/documents/{document.id!s}/attachment-upload/"
response = APIClient().post(url, {"file": file}, format="multipart")
@@ -50,14 +56,14 @@ def test_api_documents_attachment_upload_anonymous_success():
if the link reach and role permit it.
"""
document = factories.DocumentFactory(link_reach="public", link_role="editor")
file = SimpleUploadedFile("test_file.jpg", b"Dummy content")
file = SimpleUploadedFile(name="test.png", content=PIXEL, content_type="image/png")
url = f"/api/v1.0/documents/{document.id!s}/attachment-upload/"
response = APIClient().post(url, {"file": file}, format="multipart")
assert response.status_code == 201
pattern = re.compile(rf"^/media/{document.id!s}/attachments/(.*)\.jpg")
pattern = re.compile(rf"^/media/{document.id!s}/attachments/(.*)\.png")
match = pattern.search(response.json()["file"])
file_id = match.group(1)
@@ -79,13 +85,13 @@ def test_api_documents_attachment_upload_authenticated_forbidden(reach, role):
Users who are not related to a document can't upload attachments if the
link reach and role don't allow it.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach=reach, link_role=role)
file = SimpleUploadedFile("test_file.jpg", b"Dummy content")
file = SimpleUploadedFile(name="test.png", content=PIXEL, content_type="image/png")
url = f"/api/v1.0/documents/{document.id!s}/attachment-upload/"
response = client.post(url, {"file": file}, format="multipart")
@@ -114,14 +120,14 @@ def test_api_documents_attachment_upload_authenticated_success(reach, role):
client.force_login(user)
document = factories.DocumentFactory(link_reach=reach, link_role=role)
file = SimpleUploadedFile("test_file.jpg", b"Dummy content")
file = SimpleUploadedFile(name="test.png", content=PIXEL, content_type="image/png")
url = f"/api/v1.0/documents/{document.id!s}/attachment-upload/"
response = client.post(url, {"file": file}, format="multipart")
assert response.status_code == 201
pattern = re.compile(rf"^/media/{document.id!s}/attachments/(.*)\.jpg")
pattern = re.compile(rf"^/media/{document.id!s}/attachments/(.*)\.png")
match = pattern.search(response.json()["file"])
file_id = match.group(1)
@@ -134,7 +140,7 @@ def test_api_documents_attachment_upload_reader(via, mock_user_teams):
"""
Users who are simple readers on a document should not be allowed to upload an attachment.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -148,7 +154,7 @@ def test_api_documents_attachment_upload_reader(via, mock_user_teams):
document=document, team="lasuite", role="reader"
)
file = SimpleUploadedFile("test_file.jpg", b"Dummy content")
file = SimpleUploadedFile(name="test.png", content=PIXEL, content_type="image/png")
url = f"/api/v1.0/documents/{document.id!s}/attachment-upload/"
response = client.post(url, {"file": file}, format="multipart")
@@ -179,20 +185,28 @@ def test_api_documents_attachment_upload_success(via, role, mock_user_teams):
document=document, team="lasuite", role=role
)
file = SimpleUploadedFile("test_file.jpg", b"Dummy content")
file = SimpleUploadedFile(name="test.png", content=PIXEL, content_type="image/png")
url = f"/api/v1.0/documents/{document.id!s}/attachment-upload/"
response = client.post(url, {"file": file}, format="multipart")
assert response.status_code == 201
pattern = re.compile(rf"^/media/{document.id!s}/attachments/(.*)\.jpg")
match = pattern.search(response.json()["file"])
file_path = response.json()["file"]
pattern = re.compile(rf"^/media/{document.id!s}/attachments/(.*)\.png")
match = pattern.search(file_path)
file_id = match.group(1)
# Validate that file_id is a valid UUID
uuid.UUID(file_id)
# Now, check the metadata of the uploaded file
key = file_path.replace("/media", "")
file_head = default_storage.connection.meta.client.head_object(
Bucket=default_storage.bucket_name, Key=key
)
assert file_head["Metadata"] == {"owner": str(user.id)}
def test_api_documents_attachment_upload_invalid(client):
"""Attempt to upload without a file should return an explicit error."""
@@ -222,8 +236,9 @@ def test_api_documents_attachment_upload_size_limit_exceeded(settings):
url = f"/api/v1.0/documents/{document.id!s}/attachment-upload/"
# Create a temporary file larger than the allowed size
content = b"a" * (1048576 + 1)
file = ContentFile(content, name="test.jpg")
file = SimpleUploadedFile(
name="test.txt", content=b"a" * (1048576 + 1), content_type="text/plain"
)
response = client.post(url, {"file": file}, format="multipart")
@@ -231,10 +246,21 @@ def test_api_documents_attachment_upload_size_limit_exceeded(settings):
assert response.json() == {"file": ["File size exceeds the maximum limit of 1 MB."]}
def test_api_documents_attachment_upload_type_not_allowed(settings):
"""The uploaded file should be of a whitelisted type."""
settings.DOCUMENT_IMAGE_ALLOWED_MIME_TYPES = ["image/jpeg", "image/png"]
@pytest.mark.parametrize(
"name,content,extension",
[
("test.exe", b"text", "exe"),
("test", b"text", "txt"),
("test.aaaaaa", b"test", "txt"),
("test.txt", PIXEL, "txt"),
("test.py", b"#!/usr/bin/python", "py"),
],
)
def test_api_documents_attachment_upload_fix_extension(name, content, extension):
"""
A file with no extension or a wrong extension is accepted and the extension
is corrected in storage.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
@@ -242,14 +268,70 @@ def test_api_documents_attachment_upload_type_not_allowed(settings):
document = factories.DocumentFactory(users=[(user, "owner")])
url = f"/api/v1.0/documents/{document.id!s}/attachment-upload/"
# Create a temporary file with a not allowed type (e.g., text file)
file = ContentFile(b"a" * 1048576, name="test.txt")
file = SimpleUploadedFile(name=name, content=content)
response = client.post(url, {"file": file}, format="multipart")
assert response.status_code == 201
file_path = response.json()["file"]
pattern = re.compile(rf"^/media/{document.id!s}/attachments/(.*)\.{extension:s}")
match = pattern.search(file_path)
file_id = match.group(1)
# Validate that file_id is a valid UUID
uuid.UUID(file_id)
# Now, check the metadata of the uploaded file
key = file_path.replace("/media", "")
file_head = default_storage.connection.meta.client.head_object(
Bucket=default_storage.bucket_name, Key=key
)
assert file_head["Metadata"] == {"owner": str(user.id), "is_unsafe": "true"}
def test_api_documents_attachment_upload_empty_file():
"""An empty file should be rejected."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(users=[(user, "owner")])
url = f"/api/v1.0/documents/{document.id!s}/attachment-upload/"
file = SimpleUploadedFile(name="test.png", content=b"")
response = client.post(url, {"file": file}, format="multipart")
assert response.status_code == 400
assert response.json() == {
"file": [
"File type 'text/plain' is not allowed. Allowed types are: image/jpeg, image/png"
]
}
assert response.json() == {"file": ["The submitted file is empty."]}
def test_api_documents_attachment_upload_unsafe():
"""A file with an unsafe mime type should be tagged as such."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(users=[(user, "owner")])
url = f"/api/v1.0/documents/{document.id!s}/attachment-upload/"
file = SimpleUploadedFile(
name="script.exe", content=b"\x4d\x5a\x90\x00\x03\x00\x00\x00"
)
response = client.post(url, {"file": file}, format="multipart")
assert response.status_code == 201
file_path = response.json()["file"]
pattern = re.compile(rf"^/media/{document.id!s}/attachments/(.*)\.exe")
match = pattern.search(file_path)
file_id = match.group(1)
# Validate that file_id is a valid UUID
uuid.UUID(file_id)
# Now, check the metadata of the uploaded file
key = file_path.replace("/media", "")
file_head = default_storage.connection.meta.client.head_object(
Bucket=default_storage.bucket_name, Key=key
)
assert file_head["Metadata"] == {"owner": str(user.id), "is_unsafe": "true"}

View File

@@ -30,7 +30,7 @@ def test_api_documents_delete_authenticated_unrelated(reach, role):
Authenticated users should not be allowed to delete a document to which
they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -42,7 +42,7 @@ def test_api_documents_delete_authenticated_unrelated(reach, role):
)
assert response.status_code == 403
assert models.Document.objects.count() == 1
assert models.Document.objects.count() == 2
@pytest.mark.parametrize("role", ["reader", "editor", "administrator"])
@@ -52,7 +52,7 @@ def test_api_documents_delete_authenticated_not_owner(via, role, mock_user_teams
Authenticated users should not be allowed to delete a document for which they are
only a reader, editor or administrator.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -74,7 +74,7 @@ def test_api_documents_delete_authenticated_not_owner(via, role, mock_user_teams
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
assert models.Document.objects.count() == 1
assert models.Document.objects.count() == 2
@pytest.mark.parametrize("via", VIA)

View File

@@ -42,7 +42,7 @@ def test_api_documents_link_configuration_update_authenticated_unrelated(reach,
Authenticated users should not be allowed to update the link configuration for
a document to which they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -78,7 +78,7 @@ def test_api_documents_link_configuration_update_authenticated_related_forbidden
Users who are readers or editors of a document should not be allowed to update
the link configuration.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)

View File

@@ -21,6 +21,8 @@ def test_api_documents_retrieve_anonymous_public():
assert response.json() == {
"id": str(document.id),
"abilities": {
"ai_transform": document.link_role == "editor",
"ai_translate": document.link_role == "editor",
"attachment_upload": document.link_role == "editor",
"destroy": False,
"link_configuration": False,
@@ -75,6 +77,8 @@ def test_api_documents_retrieve_authenticated_unrelated_public_or_authenticated(
assert response.json() == {
"id": str(document.id),
"abilities": {
"ai_transform": document.link_role == "editor",
"ai_translate": document.link_role == "editor",
"attachment_upload": document.link_role == "editor",
"link_configuration": False,
"destroy": False,
@@ -133,7 +137,7 @@ def test_api_documents_retrieve_authenticated_unrelated_restricted():
Authenticated users should not be allowed to retrieve a document that is restricted and
to which they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -208,7 +212,7 @@ def test_api_documents_retrieve_authenticated_related_team_none(mock_user_teams)
"""
mock_user_teams.return_value = []
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)

View File

@@ -142,7 +142,7 @@ def test_api_documents_retrieve_auth_authenticated_restricted():
"""
document = factories.DocumentFactory(link_reach="restricted")
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)

View File

@@ -66,14 +66,14 @@ def test_api_documents_update_authenticated_unrelated_forbidden(reach, role):
Authenticated users should not be allowed to update a document to which
they are not related if the link configuration does not allow it.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach=reach, link_role=role)
old_document_values = serializers.DocumentSerializer(instance=document).data
old_document_values = serializers.DocumentSerializer(instance=document).data
new_document_values = serializers.DocumentSerializer(
instance=factories.DocumentFactory()
).data
@@ -111,14 +111,14 @@ def test_api_documents_update_anonymous_or_authenticated_unrelated(
client = APIClient()
if is_authenticated:
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client.force_login(user)
else:
user = AnonymousUser()
document = factories.DocumentFactory(link_reach=reach, link_role=role)
old_document_values = serializers.DocumentSerializer(instance=document).data
old_document_values = serializers.DocumentSerializer(instance=document).data
new_document_values = serializers.DocumentSerializer(
instance=factories.DocumentFactory()
).data
@@ -146,7 +146,7 @@ def test_api_documents_update_authenticated_reader(via, mock_user_teams):
Users who are reader of a document but not administrators should
not be allowed to update it.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -187,7 +187,7 @@ def test_api_documents_update_authenticated_editor_administrator_or_owner(
via, role, mock_user_teams
):
"""A user who is editor, administrator or owner of a document should be allowed to update it."""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -227,7 +227,7 @@ def test_api_documents_update_authenticated_editor_administrator_or_owner(
@pytest.mark.parametrize("via", VIA)
def test_api_documents_update_authenticated_owners(via, mock_user_teams):
"""Administrators of a document should be allowed to update it."""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
@@ -269,7 +269,7 @@ def test_api_documents_update_administrator_or_owner_of_another(via, mock_user_t
Being administrator or owner of a document should not grant authorization to update
another document.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)

View File

@@ -32,7 +32,7 @@ def test_api_template_accesses_list_authenticated_unrelated():
Authenticated users should not be allowed to list template accesses for a template
to which they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
@@ -62,7 +62,7 @@ def test_api_template_accesses_list_authenticated_related(via, mock_user_teams):
Authenticated users should be able to list template accesses for a template
to which they are directly related, whatever their role in the template.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
@@ -146,7 +146,7 @@ def test_api_template_accesses_retrieve_authenticated_unrelated():
Authenticated users should not be allowed to retrieve a template access for
a template to which they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
@@ -183,7 +183,7 @@ def test_api_template_accesses_retrieve_authenticated_related(via, mock_user_tea
A user who is related to a template should be allowed to retrieve the
associated template user accesses.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
@@ -211,199 +211,6 @@ def test_api_template_accesses_retrieve_authenticated_related(via, mock_user_tea
}
def test_api_template_accesses_create_anonymous():
"""Anonymous users should not be allowed to create template accesses."""
user = factories.UserFactory()
template = factories.TemplateFactory()
response = APIClient().post(
f"/api/v1.0/templates/{template.id!s}/accesses/",
{
"user": str(user.id),
"template": str(template.id),
"role": random.choice(models.RoleChoices.choices)[0],
},
format="json",
)
assert response.status_code == 401
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
assert models.TemplateAccess.objects.exists() is False
def test_api_template_accesses_create_authenticated_unrelated():
"""
Authenticated users should not be allowed to create template accesses for a template to
which they are not related.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
other_user = factories.UserFactory()
template = factories.TemplateFactory()
response = client.post(
f"/api/v1.0/templates/{template.id!s}/accesses/",
{
"user": str(other_user.id),
},
format="json",
)
assert response.status_code == 403
assert not models.TemplateAccess.objects.filter(user=other_user).exists()
@pytest.mark.parametrize("role", ["reader", "editor"])
@pytest.mark.parametrize("via", VIA)
def test_api_template_accesses_create_authenticated_editor_or_reader(
via, role, mock_user_teams
):
"""Editors or readers of a template should not be allowed to create template accesses."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
template = factories.TemplateFactory()
if via == USER:
factories.UserTemplateAccessFactory(template=template, user=user, role=role)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamTemplateAccessFactory(
template=template, team="lasuite", role=role
)
other_user = factories.UserFactory()
for new_role in [role[0] for role in models.RoleChoices.choices]:
response = client.post(
f"/api/v1.0/templates/{template.id!s}/accesses/",
{
"user": str(other_user.id),
"role": new_role,
},
format="json",
)
assert response.status_code == 403
assert not models.TemplateAccess.objects.filter(user=other_user).exists()
@pytest.mark.parametrize("via", VIA)
def test_api_template_accesses_create_authenticated_administrator(via, mock_user_teams):
"""
Administrators of a template should be able to create template accesses
except for the "owner" role.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
template = factories.TemplateFactory()
if via == USER:
factories.UserTemplateAccessFactory(
template=template, user=user, role="administrator"
)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamTemplateAccessFactory(
template=template, team="lasuite", role="administrator"
)
other_user = factories.UserFactory()
# It should not be allowed to create an owner access
response = client.post(
f"/api/v1.0/templates/{template.id!s}/accesses/",
{
"user": str(other_user.id),
"role": "owner",
},
format="json",
)
assert response.status_code == 403
assert response.json() == {
"detail": "Only owners of a resource can assign other users as owners."
}
# It should be allowed to create a lower access
role = random.choice(
[role[0] for role in models.RoleChoices.choices if role[0] != "owner"]
)
response = client.post(
f"/api/v1.0/templates/{template.id!s}/accesses/",
{
"user": str(other_user.id),
"role": role,
},
format="json",
)
assert response.status_code == 201
assert models.TemplateAccess.objects.filter(user=other_user).count() == 1
new_template_access = models.TemplateAccess.objects.filter(user=other_user).get()
assert response.json() == {
"abilities": new_template_access.get_abilities(user),
"id": str(new_template_access.id),
"team": "",
"role": role,
"user": str(other_user.id),
}
@pytest.mark.parametrize("via", VIA)
def test_api_template_accesses_create_authenticated_owner(via, mock_user_teams):
"""
Owners of a template should be able to create template accesses whatever the role.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
template = factories.TemplateFactory()
if via == USER:
factories.UserTemplateAccessFactory(template=template, user=user, role="owner")
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamTemplateAccessFactory(
template=template, team="lasuite", role="owner"
)
other_user = factories.UserFactory()
role = random.choice([role[0] for role in models.RoleChoices.choices])
response = client.post(
f"/api/v1.0/templates/{template.id!s}/accesses/",
{
"user": str(other_user.id),
"role": role,
},
format="json",
)
assert response.status_code == 201
assert models.TemplateAccess.objects.filter(user=other_user).count() == 1
new_template_access = models.TemplateAccess.objects.filter(user=other_user).get()
assert response.json() == {
"id": str(new_template_access.id),
"user": str(other_user.id),
"team": "",
"role": role,
"abilities": new_template_access.get_abilities(user),
}
def test_api_template_accesses_update_anonymous():
"""Anonymous users should not be allowed to update a template access."""
access = factories.UserTemplateAccessFactory()
@@ -434,14 +241,14 @@ def test_api_template_accesses_update_authenticated_unrelated():
Authenticated users should not be allowed to update a template access for a template to which
they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
access = factories.UserTemplateAccessFactory()
old_values = serializers.TemplateAccessSerializer(instance=access).data
old_values = serializers.TemplateAccessSerializer(instance=access).data
new_values = {
"id": uuid4(),
"user": factories.UserFactory().id,
@@ -467,7 +274,7 @@ def test_api_template_accesses_update_authenticated_editor_or_reader(
via, role, mock_user_teams
):
"""Editors or readers of a template should not be allowed to update its accesses."""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
@@ -509,7 +316,7 @@ def test_api_template_accesses_update_administrator_except_owner(via, mock_user_
A user who is a direct administrator in a template should be allowed to update a user
access for this template, as long as they don't try to set the role to owner.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
@@ -529,8 +336,8 @@ def test_api_template_accesses_update_administrator_except_owner(via, mock_user_
template=template,
role=random.choice(["administrator", "editor", "reader"]),
)
old_values = serializers.TemplateAccessSerializer(instance=access).data
old_values = serializers.TemplateAccessSerializer(instance=access).data
new_values = {
"id": uuid4(),
"user_id": factories.UserFactory().id,
@@ -566,7 +373,7 @@ def test_api_template_accesses_update_administrator_from_owner(via, mock_user_te
A user who is an administrator in a template, should not be allowed to update
the user access of an "owner" for this template.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
@@ -586,8 +393,8 @@ def test_api_template_accesses_update_administrator_from_owner(via, mock_user_te
access = factories.UserTemplateAccessFactory(
template=template, user=other_user, role="owner"
)
old_values = serializers.TemplateAccessSerializer(instance=access).data
old_values = serializers.TemplateAccessSerializer(instance=access).data
new_values = {
"id": uuid4(),
"user_id": factories.UserFactory().id,
@@ -613,7 +420,7 @@ def test_api_template_accesses_update_administrator_to_owner(via, mock_user_team
A user who is an administrator in a template, should not be allowed to update
the user access of another user to grant template ownership.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
@@ -635,8 +442,8 @@ def test_api_template_accesses_update_administrator_to_owner(via, mock_user_team
user=other_user,
role=random.choice(["administrator", "editor", "reader"]),
)
old_values = serializers.TemplateAccessSerializer(instance=access).data
old_values = serializers.TemplateAccessSerializer(instance=access).data
new_values = {
"id": uuid4(),
"user_id": factories.UserFactory().id,
@@ -667,7 +474,7 @@ def test_api_template_accesses_update_owner(via, mock_user_teams):
A user who is an owner in a template should be allowed to update
a user access for this template whatever the role.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
@@ -685,8 +492,8 @@ def test_api_template_accesses_update_owner(via, mock_user_teams):
access = factories.UserTemplateAccessFactory(
template=template,
)
old_values = serializers.TemplateAccessSerializer(instance=access).data
old_values = serializers.TemplateAccessSerializer(instance=access).data
new_values = {
"id": uuid4(),
"user_id": factories.UserFactory().id,
@@ -723,22 +530,21 @@ def test_api_template_accesses_update_owner_self(via, mock_user_teams):
A user who is owner of a template should be allowed to update
their own user access provided there are other owners in the template.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
template = factories.TemplateFactory()
access = None
if via == USER:
access = factories.UserTemplateAccessFactory(
template=template, user=user, role="owner"
)
elif via == TEAM:
if via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
access = factories.TeamTemplateAccessFactory(
template=template, team="lasuite", role="owner"
)
else:
access = factories.UserTemplateAccessFactory(
template=template, user=user, role="owner"
)
old_values = serializers.TemplateAccessSerializer(instance=access).data
new_role = random.choice(["administrator", "editor", "reader"])
@@ -787,7 +593,7 @@ def test_api_template_accesses_delete_authenticated():
Authenticated users should not be allowed to delete a template access for a
template to which they are not related.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
@@ -799,7 +605,7 @@ def test_api_template_accesses_delete_authenticated():
)
assert response.status_code == 403
assert models.TemplateAccess.objects.count() == 1
assert models.TemplateAccess.objects.count() == 2
@pytest.mark.parametrize("role", ["reader", "editor"])
@@ -809,7 +615,7 @@ def test_api_template_accesses_delete_editor_or_reader(via, role, mock_user_team
Authenticated users should not be allowed to delete a template access for a
template in which they are a simple editor or reader.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
@@ -825,7 +631,7 @@ def test_api_template_accesses_delete_editor_or_reader(via, role, mock_user_team
access = factories.UserTemplateAccessFactory(template=template)
assert models.TemplateAccess.objects.count() == 2
assert models.TemplateAccess.objects.count() == 3
assert models.TemplateAccess.objects.filter(user=access.user).exists()
response = client.delete(
@@ -833,7 +639,7 @@ def test_api_template_accesses_delete_editor_or_reader(via, role, mock_user_team
)
assert response.status_code == 403
assert models.TemplateAccess.objects.count() == 2
assert models.TemplateAccess.objects.count() == 3
@pytest.mark.parametrize("via", VIA)
@@ -881,7 +687,7 @@ def test_api_template_accesses_delete_administrator_on_owners(via, mock_user_tea
Users who are administrators in a template should not be allowed to delete an ownership
access from the template.
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
@@ -899,7 +705,7 @@ def test_api_template_accesses_delete_administrator_on_owners(via, mock_user_tea
access = factories.UserTemplateAccessFactory(template=template, role="owner")
assert models.TemplateAccess.objects.count() == 2
assert models.TemplateAccess.objects.count() == 3
assert models.TemplateAccess.objects.filter(user=access.user).exists()
response = client.delete(
@@ -907,7 +713,7 @@ def test_api_template_accesses_delete_administrator_on_owners(via, mock_user_tea
)
assert response.status_code == 403
assert models.TemplateAccess.objects.count() == 2
assert models.TemplateAccess.objects.count() == 3
@pytest.mark.parametrize("via", VIA)
@@ -948,7 +754,7 @@ def test_api_template_accesses_delete_owners_last_owner(via, mock_user_teams):
"""
It should not be possible to delete the last owner access from a template
"""
user = factories.UserFactory()
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
@@ -965,10 +771,10 @@ def test_api_template_accesses_delete_owners_last_owner(via, mock_user_teams):
template=template, team="lasuite", role="owner"
)
assert models.TemplateAccess.objects.count() == 1
assert models.TemplateAccess.objects.count() == 2
response = client.delete(
f"/api/v1.0/templates/{template.id!s}/accesses/{access.id!s}/",
)
assert response.status_code == 403
assert models.TemplateAccess.objects.count() == 1
assert models.TemplateAccess.objects.count() == 2

View File

@@ -0,0 +1,206 @@
"""
Test template accesses create API endpoint for users in impress's core app.
"""
import random
import pytest
from rest_framework.test import APIClient
from core import factories, models
from core.tests.conftest import TEAM, USER, VIA
pytestmark = pytest.mark.django_db
def test_api_template_accesses_create_anonymous():
"""Anonymous users should not be allowed to create template accesses."""
template = factories.TemplateFactory()
other_user = factories.UserFactory()
response = APIClient().post(
f"/api/v1.0/templates/{template.id!s}/accesses/",
{
"user": str(other_user.id),
"template": str(template.id),
"role": random.choice(models.RoleChoices.choices)[0],
},
format="json",
)
assert response.status_code == 401
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
assert models.TemplateAccess.objects.exists() is False
def test_api_template_accesses_create_authenticated_unrelated():
"""
Authenticated users should not be allowed to create template accesses for a template to
which they are not related.
"""
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
other_user = factories.UserFactory()
template = factories.TemplateFactory()
response = client.post(
f"/api/v1.0/templates/{template.id!s}/accesses/",
{
"user": str(other_user.id),
},
format="json",
)
assert response.status_code == 403
assert not models.TemplateAccess.objects.filter(user=other_user).exists()
@pytest.mark.parametrize("role", ["reader", "editor"])
@pytest.mark.parametrize("via", VIA)
def test_api_template_accesses_create_authenticated_editor_or_reader(
via, role, mock_user_teams
):
"""Editors or readers of a template should not be allowed to create template accesses."""
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
template = factories.TemplateFactory()
if via == USER:
factories.UserTemplateAccessFactory(template=template, user=user, role=role)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamTemplateAccessFactory(
template=template, team="lasuite", role=role
)
other_user = factories.UserFactory()
for new_role in [role[0] for role in models.RoleChoices.choices]:
response = client.post(
f"/api/v1.0/templates/{template.id!s}/accesses/",
{
"user": str(other_user.id),
"role": new_role,
},
format="json",
)
assert response.status_code == 403
assert not models.TemplateAccess.objects.filter(user=other_user).exists()
@pytest.mark.parametrize("via", VIA)
def test_api_template_accesses_create_authenticated_administrator(via, mock_user_teams):
"""
Administrators of a template should be able to create template accesses
except for the "owner" role.
"""
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
template = factories.TemplateFactory()
if via == USER:
factories.UserTemplateAccessFactory(
template=template, user=user, role="administrator"
)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamTemplateAccessFactory(
template=template, team="lasuite", role="administrator"
)
other_user = factories.UserFactory()
# It should not be allowed to create an owner access
response = client.post(
f"/api/v1.0/templates/{template.id!s}/accesses/",
{
"user": str(other_user.id),
"role": "owner",
},
format="json",
)
assert response.status_code == 403
assert response.json() == {
"detail": "Only owners of a resource can assign other users as owners."
}
# It should be allowed to create a lower access
role = random.choice(
[role[0] for role in models.RoleChoices.choices if role[0] != "owner"]
)
response = client.post(
f"/api/v1.0/templates/{template.id!s}/accesses/",
{
"user": str(other_user.id),
"role": role,
},
format="json",
)
assert response.status_code == 201
assert models.TemplateAccess.objects.filter(user=other_user).count() == 1
new_template_access = models.TemplateAccess.objects.filter(user=other_user).get()
assert response.json() == {
"abilities": new_template_access.get_abilities(user),
"id": str(new_template_access.id),
"team": "",
"role": role,
"user": str(other_user.id),
}
@pytest.mark.parametrize("via", VIA)
def test_api_template_accesses_create_authenticated_owner(via, mock_user_teams):
"""
Owners of a template should be able to create template accesses whatever the role.
"""
user = factories.UserFactory(with_owned_template=True)
client = APIClient()
client.force_login(user)
template = factories.TemplateFactory()
if via == USER:
factories.UserTemplateAccessFactory(template=template, user=user, role="owner")
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamTemplateAccessFactory(
template=template, team="lasuite", role="owner"
)
other_user = factories.UserFactory()
role = random.choice([role[0] for role in models.RoleChoices.choices])
response = client.post(
f"/api/v1.0/templates/{template.id!s}/accesses/",
{
"user": str(other_user.id),
"role": role,
},
format="json",
)
assert response.status_code == 201
assert models.TemplateAccess.objects.filter(user=other_user).count() == 1
new_template_access = models.TemplateAccess.objects.filter(user=other_user).get()
assert response.json() == {
"id": str(new_template_access.id),
"user": str(other_user.id),
"team": "",
"role": role,
"abilities": new_template_access.get_abilities(user),
}

View File

@@ -0,0 +1,127 @@
"""
Test throttling on documents for the AI endpoint.
"""
from unittest.mock import patch
from django.core.cache import cache
from django.test import override_settings
import pytest
from rest_framework.response import Response
from rest_framework.test import APIRequestFactory
from rest_framework.views import APIView
from core.api.utils import AIDocumentRateThrottle
class DocumentAPIView(APIView):
"""A simple view to test the throttle"""
throttle_classes = [AIDocumentRateThrottle]
def get(self, request, *args, **kwargs):
"""Minimal get method for testing purposes."""
return Response({"message": "Success"})
@pytest.fixture(autouse=True)
def clear_cache():
"""Fixture to clear the cache before each test."""
cache.clear()
@override_settings(AI_DOCUMENT_RATE_THROTTLE_RATES={"minute": 3, "hour": 6, "day": 10})
@patch("time.time")
def test_api_utils_ai_document_rate_throttle_minute_limit(mock_time):
"""Test that minute limit is enforced."""
api_rf = APIRequestFactory()
mock_time.return_value = 1000000
# Simulate requests to the document API
for _i in range(3): # 3 first requests should be allowed
request = api_rf.get("/documents/1/")
response = DocumentAPIView.as_view()(request, pk=1)
assert response.status_code == 200
# Simulate passage of time
mock_time.return_value += 59
# 4th request should be throttled
request = api_rf.get("/documents/1/")
response = DocumentAPIView.as_view()(request, pk=1)
assert response.status_code == 429
# After the 60s backoff wait time has passed, we can make a request again
mock_time.return_value += 1
request = api_rf.get("/documents/1/")
response = DocumentAPIView.as_view()(request, pk=1)
assert response.status_code == 200
@override_settings(
AI_DOCUMENT_RATE_THROTTLE_RATES={"minute": 100000, "hour": 6, "day": 10}
)
@patch("time.time")
def test_ai_document_rate_throttle_hour_limit(mock_time):
"""Test that the hour limit is enforced without hitting the minute limit."""
api_rf = APIRequestFactory()
mock_time.return_value = 1000000
# Make requests to the document API, one per 21 seconds to avoid hitting the minute limit
for _i in range(6):
request = api_rf.get("/documents/1/")
response = DocumentAPIView.as_view()(request, pk=1)
assert response.status_code == 200
# Simulate passage of time
mock_time.return_value += 21
# Simulate passage of time
mock_time.return_value += 3600 - 6 * 21 - 1
# 7th request should be throttled
request = api_rf.get("/documents/1/")
response = DocumentAPIView.as_view()(request, pk=1)
assert response.status_code == 429
# After the 1h backoff wait time has passed, we can make a request again
mock_time.return_value += 1
request = api_rf.get("/documents/1/")
response = DocumentAPIView.as_view()(request, pk=1)
assert response.status_code == 200
@override_settings(AI_DOCUMENT_RATE_THROTTLE_RATES={"minute": 3, "hour": 6, "day": 10})
@patch("time.time")
def test_api_utils_ai_document_rate_throttle_day_limit(mock_time):
"""Test that day limit is enforced."""
api_rf = APIRequestFactory()
mock_time.return_value = 1000000
# Make requests to the document API, one per 10 minutes to avoid hitting
# the minute and hour limits
for _i in range(10): # 10 requests should be allowed
request = api_rf.get("/documents/1/")
response = DocumentAPIView.as_view()(request, pk=1)
assert response.status_code == 200
# Simulate passage of time
mock_time.return_value += 60 * 10
# Simulate passage of time
mock_time.return_value += 24 * 3600 - 10 * 60 * 10 - 1
# 11th request should be throttled
request = api_rf.get("/documents/1/")
response = DocumentAPIView.as_view()(request, pk=1)
assert response.status_code == 429
# After the 24h backoff wait time has passed we can make a request again
mock_time.return_value += 1
request = api_rf.get("/documents/1/")
response = DocumentAPIView.as_view()(request, pk=1)
assert response.status_code == 200

View File

@@ -0,0 +1,146 @@
"""
Test throttling on users for the AI endpoint.
"""
from unittest.mock import patch
from uuid import uuid4
from django.core.cache import cache
from django.test import override_settings
import pytest
from rest_framework.response import Response
from rest_framework.test import APIRequestFactory
from rest_framework.views import APIView
from core.api.utils import AIUserRateThrottle
from core.factories import UserFactory
pytestmark = pytest.mark.django_db
class DocumentAPIView(APIView):
"""A simple view to test the throttle"""
throttle_classes = [AIUserRateThrottle]
def get(self, request, *args, **kwargs):
"""Minimal get method for testing purposes."""
return Response({"message": "Success"})
@pytest.fixture(autouse=True)
def clear_cache():
"""Fixture to clear the cache before each test."""
cache.clear()
@override_settings(AI_USER_RATE_THROTTLE_RATES={"minute": 3, "hour": 6, "day": 10})
@patch("time.time")
def test_api_utils_ai_user_rate_throttle_minute_limit(mock_time):
"""Test that minute limit is enforced."""
user = UserFactory()
api_rf = APIRequestFactory()
mock_time.return_value = 1000000
# Simulate requests to the document API
for _i in range(3): # 3 first requests should be allowed
document_id = str(uuid4())
request = api_rf.get(f"/documents/{document_id:s}/")
request.user = user
response = DocumentAPIView.as_view()(request, pk=document_id)
assert response.status_code == 200
# Simulate passage of time
mock_time.return_value += 59
# 4th request should be throttled
document_id = str(uuid4())
request = api_rf.get(f"/documents/{document_id:s}/")
request.user = user
response = DocumentAPIView.as_view()(request, pk=document_id)
assert response.status_code == 429
# After the 60s backoff wait time has passed, we can make a request again
mock_time.return_value += 1
document_id = str(uuid4())
request = api_rf.get(f"/documents/{document_id:s}/")
request.user = user
response = DocumentAPIView.as_view()(request, pk=document_id)
assert response.status_code == 200
@override_settings(AI_USER_RATE_THROTTLE_RATES={"minute": 100000, "hour": 6, "day": 10})
@patch("time.time")
def test_ai_user_rate_throttle_hour_limit(mock_time):
"""Test that the hour limit is enforced without hitting the minute limit."""
user = UserFactory()
api_rf = APIRequestFactory()
mock_time.return_value = 1000000
# Make requests to the document API, one per 21 seconds to avoid hitting the minute limit
for _i in range(6):
document_id = str(uuid4())
request = api_rf.get(f"/documents/{document_id:s}/")
request.user = user
response = DocumentAPIView.as_view()(request, pk=document_id)
assert response.status_code == 200
# Simulate passage of time
mock_time.return_value += 21
# Simulate passage of time
mock_time.return_value += 3600 - 6 * 21 - 1
# 7th request should be throttled
request = api_rf.get(f"/documents/{document_id:s}/")
request.user = user
response = DocumentAPIView.as_view()(request, pk=document_id)
assert response.status_code == 429
# After the 1h backoff wait time has passed, we can make a request again
mock_time.return_value += 1
request = api_rf.get(f"/documents/{document_id:s}/")
request.user = user
response = DocumentAPIView.as_view()(request, pk=document_id)
assert response.status_code == 200
@override_settings(AI_USER_RATE_THROTTLE_RATES={"minute": 3, "hour": 6, "day": 10})
@patch("time.time")
def test_api_utils_ai_user_rate_throttle_day_limit(mock_time):
"""Test that day limit is enforced."""
user = UserFactory()
api_rf = APIRequestFactory()
mock_time.return_value = 1000000
# Make requests to the document API, one per 10 minutes to avoid hitting
# the minute and hour limits
for _i in range(10): # 10 requests should be allowed
document_id = str(uuid4())
request = api_rf.get(f"/documents/{document_id:s}/")
request.user = user
response = DocumentAPIView.as_view()(request, pk=document_id)
assert response.status_code == 200
# Simulate passage of time
mock_time.return_value += 60 * 10
# Simulate passage of time
mock_time.return_value += 24 * 3600 - 10 * 60 * 10 - 1
# 11th request should be throttled
request = api_rf.get(f"/documents/{document_id:s}/")
request.user = user
response = DocumentAPIView.as_view()(request, pk=document_id)
assert response.status_code == 429
# After the 24h backoff wait time has passed we can make a request again
mock_time.return_value += 1
request = api_rf.get(f"/documents/{document_id:s}/")
request.user = user
response = DocumentAPIView.as_view()(request, pk=document_id)
assert response.status_code == 200

View File

@@ -83,6 +83,8 @@ def test_models_documents_get_abilities_forbidden(is_authenticated, reach, role)
user = factories.UserFactory() if is_authenticated else AnonymousUser()
abilities = document.get_abilities(user)
assert abilities == {
"ai_transform": False,
"ai_translate": False,
"attachment_upload": False,
"link_configuration": False,
"destroy": False,
@@ -113,6 +115,8 @@ def test_models_documents_get_abilities_reader(is_authenticated, reach):
user = factories.UserFactory() if is_authenticated else AnonymousUser()
abilities = document.get_abilities(user)
assert abilities == {
"ai_transform": False,
"ai_translate": False,
"attachment_upload": False,
"destroy": False,
"link_configuration": False,
@@ -143,6 +147,8 @@ def test_models_documents_get_abilities_editor(is_authenticated, reach):
user = factories.UserFactory() if is_authenticated else AnonymousUser()
abilities = document.get_abilities(user)
assert abilities == {
"ai_transform": True,
"ai_translate": True,
"attachment_upload": True,
"destroy": False,
"link_configuration": False,
@@ -162,6 +168,8 @@ def test_models_documents_get_abilities_owner():
access = factories.UserDocumentAccessFactory(role="owner", user=user)
abilities = access.document.get_abilities(access.user)
assert abilities == {
"ai_transform": True,
"ai_translate": True,
"attachment_upload": True,
"destroy": True,
"link_configuration": True,
@@ -180,6 +188,8 @@ def test_models_documents_get_abilities_administrator():
access = factories.UserDocumentAccessFactory(role="administrator")
abilities = access.document.get_abilities(access.user)
assert abilities == {
"ai_transform": True,
"ai_translate": True,
"attachment_upload": True,
"destroy": False,
"link_configuration": True,
@@ -201,6 +211,8 @@ def test_models_documents_get_abilities_editor_user(django_assert_num_queries):
abilities = access.document.get_abilities(access.user)
assert abilities == {
"ai_transform": True,
"ai_translate": True,
"attachment_upload": True,
"destroy": False,
"link_configuration": False,
@@ -224,6 +236,8 @@ def test_models_documents_get_abilities_reader_user(django_assert_num_queries):
abilities = access.document.get_abilities(access.user)
assert abilities == {
"ai_transform": False,
"ai_translate": False,
"attachment_upload": False,
"destroy": False,
"link_configuration": False,
@@ -248,6 +262,8 @@ def test_models_documents_get_abilities_preset_role(django_assert_num_queries):
abilities = access.document.get_abilities(access.user)
assert abilities == {
"ai_transform": False,
"ai_translate": False,
"attachment_upload": False,
"destroy": False,
"link_configuration": False,
@@ -363,8 +379,9 @@ def test_models_documents__email_invitation__success():
# pylint: disable-next=no-member
assert len(mail.outbox) == 0
sender = factories.UserFactory(full_name="Test Sender", email="sender@example.com")
document.email_invitation(
"en", "guest@example.com", models.RoleChoices.EDITOR, "sender@example.com"
"en", "guest@example.com", models.RoleChoices.EDITOR, sender
)
# pylint: disable-next=no-member
@@ -377,8 +394,8 @@ def test_models_documents__email_invitation__success():
email_content = " ".join(email.body.split())
assert (
f"sender@example.com invited you as an editor on the following document : {document.title}"
in email_content
f'Test Sender (sender@example.com) invited you with the role "editor" '
f"on the following document : {document.title}" in email_content
)
assert f"docs/{document.id}/" in email_content
@@ -392,8 +409,14 @@ def test_models_documents__email_invitation__success_fr():
# pylint: disable-next=no-member
assert len(mail.outbox) == 0
sender = factories.UserFactory(
full_name="Test Sender2", email="sender2@example.com"
)
document.email_invitation(
"fr-fr", "guest2@example.com", models.RoleChoices.OWNER, "sender2@example.com"
"fr-fr",
"guest2@example.com",
models.RoleChoices.OWNER,
sender,
)
# pylint: disable-next=no-member
@@ -406,7 +429,7 @@ def test_models_documents__email_invitation__success_fr():
email_content = " ".join(email.body.split())
assert (
f"sender2@example.com vous a invité en tant que propriétaire "
f'Test Sender2 (sender2@example.com) vous a invité avec le rôle "propriétaire" '
f"sur le document suivant : {document.title}" in email_content
)
assert f"docs/{document.id}/" in email_content
@@ -424,8 +447,12 @@ def test_models_documents__email_invitation__failed(mock_logger, _mock_send_mail
# pylint: disable-next=no-member
assert len(mail.outbox) == 0
sender = factories.UserFactory()
document.email_invitation(
"en", "guest3@example.com", models.RoleChoices.ADMIN, "sender3@example.com"
"en",
"guest3@example.com",
models.RoleChoices.ADMIN,
sender,
)
# No email has been sent

View File

@@ -0,0 +1,104 @@
"""
Test ai API endpoints in the impress core app.
"""
import json
from unittest.mock import MagicMock, patch
from django.core.exceptions import ImproperlyConfigured
from django.test.utils import override_settings
import pytest
from openai import OpenAIError
from core.services.ai_services import AIService
pytestmark = pytest.mark.django_db
@pytest.mark.parametrize(
"setting_name, setting_value",
[
("AI_BASE_URL", None),
("AI_API_KEY", None),
("AI_MODEL", None),
],
)
def test_api_ai_setting_missing(setting_name, setting_value):
"""Setting should be set"""
with override_settings(**{setting_name: setting_value}):
with pytest.raises(
ImproperlyConfigured,
match="AI configuration not set",
):
AIService()
@override_settings(
AI_BASE_URL="http://example.com", AI_API_KEY="test-key", AI_MODEL="test-model"
)
@patch("openai.resources.chat.completions.Completions.create")
def test_api_ai__client_error(mock_create):
"""Fail when the client raises an error"""
mock_create.side_effect = OpenAIError("Mocked client error")
with pytest.raises(
OpenAIError,
match="Mocked client error",
):
AIService().transform("hello", "prompt")
@override_settings(
AI_BASE_URL="http://example.com", AI_API_KEY="test-key", AI_MODEL="test-model"
)
@patch("openai.resources.chat.completions.Completions.create")
def test_api_ai__client_invalid_response(mock_create):
"""Fail when the client response is invalid"""
answer = {"no_answer": "This is an invalid response"}
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=json.dumps(answer)))]
)
with pytest.raises(
RuntimeError,
match="AI response does not contain an answer",
):
AIService().transform("hello", "prompt")
@override_settings(
AI_BASE_URL="http://example.com", AI_API_KEY="test-key", AI_MODEL="test-model"
)
@patch("openai.resources.chat.completions.Completions.create")
def test_api_ai__success(mock_create):
"""The AI request should work as expect when called with valid arguments."""
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
)
response = AIService().transform("hello", "prompt")
assert response == {"answer": "Salut"}
@override_settings(
AI_BASE_URL="http://example.com", AI_API_KEY="test-key", AI_MODEL="test-model"
)
@patch("openai.resources.chat.completions.Completions.create")
def test_api_ai__success_sanitize(mock_create):
"""The AI response should be sanitized"""
answer = '{"answer": "Salut\\n \tle \nmonde"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
)
response = AIService().transform("hello", "prompt")
assert response == {"answer": "Salut\n \tle \nmonde"}

View File

@@ -5,7 +5,7 @@ from django.urls import include, path, re_path
from rest_framework.routers import DefaultRouter
from core.api import viewsets
from core.api import viewsets, create_summary
from core.authentication.urls import urlpatterns as oidc_urls
# - Main endpoints
@@ -44,6 +44,7 @@ urlpatterns = [
[
*router.urls,
*oidc_urls,
path("summary/", create_summary, name="create_summary"),
re_path(
r"^documents/(?P<resource_id>[0-9a-z-]*)/",
include(document_related_router.urls),

View File

@@ -144,14 +144,75 @@ class Base(Configuration):
environ_name="DOCUMENT_IMAGE_MAX_SIZE",
environ_prefix=None,
)
DOCUMENT_IMAGE_ALLOWED_MIME_TYPES = [
"image/bmp",
"image/gif",
"image/jpeg",
"image/png",
DOCUMENT_UNSAFE_MIME_TYPES = [
# Executable Files
"application/x-msdownload",
"application/x-bat",
"application/x-dosexec",
"application/x-sh",
"application/x-ms-dos-executable",
"application/x-msi",
"application/java-archive",
"application/octet-stream",
# Dynamic Web Pages
"application/x-httpd-php",
"application/x-asp",
"application/x-aspx",
"application/jsp",
"application/xhtml+xml",
"application/x-python-code",
"application/x-perl",
"text/html",
"text/javascript",
"text/x-php",
# System Files
"application/x-msdownload",
"application/x-sys",
"application/x-drv",
"application/cpl",
"application/x-apple-diskimage",
# Script Files
"application/javascript",
"application/x-vbscript",
"application/x-powershell",
"application/x-shellscript",
# Compressed/Archive Files
"application/zip",
"application/x-tar",
"application/gzip",
"application/x-bzip2",
"application/x-7z-compressed",
"application/x-rar",
"application/x-rar-compressed",
"application/x-compress",
"application/x-lzma",
# Macros in Documents
"application/vnd.ms-word",
"application/vnd.ms-excel",
"application/vnd.ms-powerpoint",
"application/vnd.ms-word.document.macroenabled.12",
"application/vnd.ms-excel.sheet.macroenabled.12",
"application/vnd.ms-powerpoint.presentation.macroenabled.12",
# Disk Images & Virtual Disk Files
"application/x-iso9660-image",
"application/x-vmdk",
"application/x-apple-diskimage",
"application/x-dmg",
# Other Dangerous MIME Types
"application/x-ms-application",
"application/x-msdownload",
"application/x-shockwave-flash",
"application/x-silverlight-app",
"application/x-java-vm",
"application/x-bittorrent",
"application/hta",
"application/x-csh",
"application/x-ksh",
"application/x-ms-regedit",
"application/x-msdownload",
"application/xml",
"image/svg+xml",
"image/tiff",
"image/webp",
]
# Document versions
@@ -393,6 +454,29 @@ class Base(Configuration):
ALLOW_LOGOUT_GET_METHOD = values.BooleanValue(
default=True, environ_name="ALLOW_LOGOUT_GET_METHOD", environ_prefix=None
)
AI_API_KEY = values.Value(None, environ_name="AI_API_KEY", environ_prefix=None)
AI_BASE_URL = values.Value(None, environ_name="AI_BASE_URL", environ_prefix=None)
AI_MODEL = values.Value(None, environ_name="AI_MODEL", environ_prefix=None)
AI_DOCUMENT_RATE_THROTTLE_RATES = values.DictValue(
{
"minute": 5,
"hour": 100,
"day": 500,
},
environ_name="AI_DOCUMENT_RATE_THROTTLE_RATES",
environ_prefix=None,
)
AI_USER_RATE_THROTTLE_RATES = values.DictValue(
{
"minute": 3,
"hour": 50,
"day": 200,
},
environ_name="AI_USER_RATE_THROTTLE_RATES",
environ_prefix=None,
)
USER_OIDC_FIELDS_TO_FULLNAME = values.ListValue(
default=["first_name", "last_name"],

View File

@@ -2,8 +2,8 @@ msgid ""
msgstr ""
"Project-Id-Version: lasuite-people\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2024-09-25 10:15+0000\n"
"PO-Revision-Date: 2024-09-25 10:17\n"
"POT-Creation-Date: 2024-10-15 07:19+0000\n"
"PO-Revision-Date: 2024-10-15 07:23\n"
"Last-Translator: \n"
"Language-Team: English\n"
"Language: en_US\n"
@@ -17,15 +17,15 @@ msgstr ""
"X-Crowdin-File: backend-impress.pot\n"
"X-Crowdin-File-ID: 8\n"
#: core/admin.py:32
#: core/admin.py:33
msgid "Personal info"
msgstr ""
#: core/admin.py:34
#: core/admin.py:46
msgid "Permissions"
msgstr ""
#: core/admin.py:46
#: core/admin.py:58
msgid "Important dates"
msgstr ""
@@ -41,7 +41,7 @@ msgstr ""
msgid "Format"
msgstr ""
#: core/authentication/backends.py:56
#: core/authentication/backends.py:57
msgid "Invalid response format or token verification failed"
msgstr ""
@@ -49,10 +49,6 @@ msgstr ""
msgid "User info contained no recognizable user identification"
msgstr ""
#: core/authentication/backends.py:101
msgid "Claims contained no recognizable user identification"
msgstr ""
#: core/models.py:62 core/models.py:69
msgid "Reader"
msgstr ""
@@ -117,223 +113,236 @@ msgstr ""
msgid "Required. 255 characters or fewer. Letters, numbers, and @/./+/-/_ characters only."
msgstr ""
#: core/models.py:148
#: core/models.py:149
msgid "full name"
msgstr ""
#: core/models.py:150
msgid "short name"
msgstr ""
#: core/models.py:152
msgid "identity email address"
msgstr ""
#: core/models.py:153
#: core/models.py:157
msgid "admin email address"
msgstr ""
#: core/models.py:160
#: core/models.py:164
msgid "language"
msgstr ""
#: core/models.py:161
#: core/models.py:165
msgid "The language in which the user wants to see the interface."
msgstr ""
#: core/models.py:167
#: core/models.py:171
msgid "The timezone in which the user wants to see times."
msgstr ""
#: core/models.py:170
#: core/models.py:174
msgid "device"
msgstr ""
#: core/models.py:172
#: core/models.py:176
msgid "Whether the user is a device or a real user."
msgstr ""
#: core/models.py:175
#: core/models.py:179
msgid "staff status"
msgstr ""
#: core/models.py:177
#: core/models.py:181
msgid "Whether the user can log into this admin site."
msgstr ""
#: core/models.py:180
#: core/models.py:184
msgid "active"
msgstr ""
#: core/models.py:183
#: core/models.py:187
msgid "Whether this user should be treated as active. Unselect this instead of deleting accounts."
msgstr ""
#: core/models.py:195
#: core/models.py:199
msgid "user"
msgstr ""
#: core/models.py:196
#: core/models.py:200
msgid "users"
msgstr ""
#: core/models.py:328 core/models.py:644
#: core/models.py:332 core/models.py:638
msgid "title"
msgstr ""
#: core/models.py:343
#: core/models.py:347
msgid "Document"
msgstr ""
#: core/models.py:344
#: core/models.py:348
msgid "Documents"
msgstr ""
#: core/models.py:347
#: core/models.py:351
msgid "Untitled Document"
msgstr ""
#: core/models.py:537
#: core/models.py:530
#, python-format
msgid "%(username)s shared a document with you: %(document)s"
msgid "%(sender_name)s shared a document with you: %(document)s"
msgstr ""
#: core/models.py:580
#: core/models.py:574
msgid "Document/user link trace"
msgstr ""
#: core/models.py:581
#: core/models.py:575
msgid "Document/user link traces"
msgstr ""
#: core/models.py:587
#: core/models.py:581
msgid "A link trace already exists for this document/user."
msgstr ""
#: core/models.py:608
#: core/models.py:602
msgid "Document/user relation"
msgstr ""
#: core/models.py:609
#: core/models.py:603
msgid "Document/user relations"
msgstr ""
#: core/models.py:615
#: core/models.py:609
msgid "This user is already in this document."
msgstr ""
#: core/models.py:621
#: core/models.py:615
msgid "This team is already in this document."
msgstr ""
#: core/models.py:627 core/models.py:816
#: core/models.py:621 core/models.py:810
msgid "Either user or team must be set, not both."
msgstr ""
#: core/models.py:645
#: core/models.py:639
msgid "description"
msgstr ""
#: core/models.py:646
#: core/models.py:640
msgid "code"
msgstr ""
#: core/models.py:647
#: core/models.py:641
msgid "css"
msgstr ""
#: core/models.py:649
#: core/models.py:643
msgid "public"
msgstr ""
#: core/models.py:651
#: core/models.py:645
msgid "Whether this template is public for anyone to use."
msgstr ""
#: core/models.py:657
#: core/models.py:651
msgid "Template"
msgstr ""
#: core/models.py:658
#: core/models.py:652
msgid "Templates"
msgstr ""
#: core/models.py:797
#: core/models.py:791
msgid "Template/user relation"
msgstr ""
#: core/models.py:798
#: core/models.py:792
msgid "Template/user relations"
msgstr ""
#: core/models.py:804
#: core/models.py:798
msgid "This user is already in this template."
msgstr ""
#: core/models.py:810
#: core/models.py:804
msgid "This team is already in this template."
msgstr ""
#: core/models.py:833
#: core/models.py:827
msgid "email address"
msgstr ""
#: core/models.py:850
#: core/models.py:844
msgid "Document invitation"
msgstr ""
#: core/models.py:851
#: core/models.py:845
msgid "Document invitations"
msgstr ""
#: core/models.py:868
#: core/models.py:862
msgid "This email is already associated to a registered user."
msgstr ""
#: core/templates/mail/html/invitation.html:160
#: core/templates/mail/html/invitation2.html:160
#: core/templates/mail/html/hello.html:159 core/templates/mail/text/hello.txt:3
msgid "Company logo"
msgstr ""
#: core/templates/mail/html/hello.html:188 core/templates/mail/text/hello.txt:5
#, python-format
msgid "Hello %(name)s"
msgstr ""
#: core/templates/mail/html/hello.html:188 core/templates/mail/text/hello.txt:5
msgid "Hello"
msgstr ""
#: core/templates/mail/html/hello.html:189 core/templates/mail/text/hello.txt:6
msgid "Thank you very much for your visit!"
msgstr ""
#: core/templates/mail/html/hello.html:221
#, python-format
msgid "This mail has been sent to %(email)s by <a href=\"%(href)s\">%(name)s</a>"
msgstr ""
#: core/templates/mail/html/invitation.html:159
#: core/templates/mail/text/invitation.txt:3
#: core/templates/mail/text/invitation2.txt:3
msgid "La Suite Numérique"
msgstr ""
#: core/templates/mail/html/invitation.html:190
#: core/templates/mail/html/invitation.html:189
#: core/templates/mail/text/invitation.txt:6
#, python-format
msgid " %(username)s shared a document with you ! "
msgid " %(sender_name)s shared a document with you ! "
msgstr ""
#: core/templates/mail/html/invitation.html:197
#: core/templates/mail/html/invitation.html:196
#: core/templates/mail/text/invitation.txt:8
#, python-format
msgid " %(username)s invited you as an %(role)s on the following document : "
msgid " %(sender_name_email)s invited you with the role \"%(role)s\" on the following document : "
msgstr ""
#: core/templates/mail/html/invitation.html:206
#: core/templates/mail/html/invitation2.html:211
#: core/templates/mail/html/invitation.html:205
#: core/templates/mail/text/invitation.txt:10
#: core/templates/mail/text/invitation2.txt:11
msgid "Open"
msgstr ""
#: core/templates/mail/html/invitation.html:223
#: core/templates/mail/html/invitation.html:222
#: core/templates/mail/text/invitation.txt:14
msgid " Docs, your new essential tool for organizing, sharing and collaborate on your documents as a team. "
msgid " Docs, your new essential tool for organizing, sharing and collaborating on your documents as a team. "
msgstr ""
#: core/templates/mail/html/invitation.html:230
#: core/templates/mail/html/invitation2.html:235
#: core/templates/mail/html/invitation.html:229
#: core/templates/mail/text/invitation.txt:16
#: core/templates/mail/text/invitation2.txt:17
msgid "Brought to you by La Suite Numérique"
msgstr ""
#: core/templates/mail/html/invitation2.html:190
#: core/templates/mail/text/hello.txt:8
#, python-format
msgid "%(username)s shared a document with you"
msgstr ""
#: core/templates/mail/html/invitation2.html:197
#: core/templates/mail/text/invitation2.txt:8
#, python-format
msgid "%(username)s invited you as an %(role)s on the following document :"
msgstr ""
#: core/templates/mail/html/invitation2.html:228
#: core/templates/mail/text/invitation2.txt:15
msgid "Docs, your new essential tool for organizing, sharing and collaborate on your document as a team."
msgid "This mail has been sent to %(email)s by %(name)s [%(href)s]"
msgstr ""
#: impress/settings.py:176

View File

@@ -2,8 +2,8 @@ msgid ""
msgstr ""
"Project-Id-Version: lasuite-people\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2024-09-25 10:15+0000\n"
"PO-Revision-Date: 2024-09-25 10:21\n"
"POT-Creation-Date: 2024-10-15 07:19+0000\n"
"PO-Revision-Date: 2024-10-15 07:23\n"
"Last-Translator: \n"
"Language-Team: French\n"
"Language: fr_FR\n"
@@ -17,15 +17,15 @@ msgstr ""
"X-Crowdin-File: backend-impress.pot\n"
"X-Crowdin-File-ID: 8\n"
#: core/admin.py:32
#: core/admin.py:33
msgid "Personal info"
msgstr "Infos Personnelles"
#: core/admin.py:34
#: core/admin.py:46
msgid "Permissions"
msgstr "Permissions"
#: core/admin.py:46
#: core/admin.py:58
msgid "Important dates"
msgstr "Dates importantes"
@@ -41,7 +41,7 @@ msgstr ""
msgid "Format"
msgstr ""
#: core/authentication/backends.py:56
#: core/authentication/backends.py:57
msgid "Invalid response format or token verification failed"
msgstr ""
@@ -49,10 +49,6 @@ msgstr ""
msgid "User info contained no recognizable user identification"
msgstr ""
#: core/authentication/backends.py:101
msgid "Claims contained no recognizable user identification"
msgstr ""
#: core/models.py:62 core/models.py:69
msgid "Reader"
msgstr "Lecteur"
@@ -117,224 +113,237 @@ msgstr ""
msgid "Required. 255 characters or fewer. Letters, numbers, and @/./+/-/_ characters only."
msgstr ""
#: core/models.py:148
#: core/models.py:149
msgid "full name"
msgstr ""
#: core/models.py:150
msgid "short name"
msgstr ""
#: core/models.py:152
msgid "identity email address"
msgstr ""
#: core/models.py:153
#: core/models.py:157
msgid "admin email address"
msgstr ""
#: core/models.py:160
#: core/models.py:164
msgid "language"
msgstr ""
#: core/models.py:161
#: core/models.py:165
msgid "The language in which the user wants to see the interface."
msgstr ""
#: core/models.py:167
#: core/models.py:171
msgid "The timezone in which the user wants to see times."
msgstr ""
#: core/models.py:170
#: core/models.py:174
msgid "device"
msgstr ""
#: core/models.py:172
#: core/models.py:176
msgid "Whether the user is a device or a real user."
msgstr ""
#: core/models.py:175
#: core/models.py:179
msgid "staff status"
msgstr ""
#: core/models.py:177
#: core/models.py:181
msgid "Whether the user can log into this admin site."
msgstr ""
#: core/models.py:180
#: core/models.py:184
msgid "active"
msgstr ""
#: core/models.py:183
#: core/models.py:187
msgid "Whether this user should be treated as active. Unselect this instead of deleting accounts."
msgstr ""
#: core/models.py:195
#: core/models.py:199
msgid "user"
msgstr ""
#: core/models.py:196
#: core/models.py:200
msgid "users"
msgstr ""
#: core/models.py:328 core/models.py:644
#: core/models.py:332 core/models.py:638
msgid "title"
msgstr ""
#: core/models.py:343
#: core/models.py:347
msgid "Document"
msgstr ""
#: core/models.py:344
#: core/models.py:348
msgid "Documents"
msgstr ""
#: core/models.py:347
#: core/models.py:351
msgid "Untitled Document"
msgstr ""
#: core/models.py:537
#: core/models.py:530
#, python-format
msgid "%(username)s shared a document with you: %(document)s"
msgstr "%(username)s a partagé un document avec vous: %(document)s"
msgid "%(sender_name)s shared a document with you: %(document)s"
msgstr "%(sender_name)s a partagé un document avec vous: %(document)s"
#: core/models.py:580
#: core/models.py:574
msgid "Document/user link trace"
msgstr ""
#: core/models.py:581
#: core/models.py:575
msgid "Document/user link traces"
msgstr ""
#: core/models.py:587
#: core/models.py:581
msgid "A link trace already exists for this document/user."
msgstr ""
#: core/models.py:608
#: core/models.py:602
msgid "Document/user relation"
msgstr ""
#: core/models.py:609
#: core/models.py:603
msgid "Document/user relations"
msgstr ""
#: core/models.py:615
#: core/models.py:609
msgid "This user is already in this document."
msgstr ""
#: core/models.py:621
#: core/models.py:615
msgid "This team is already in this document."
msgstr ""
#: core/models.py:627 core/models.py:816
#: core/models.py:621 core/models.py:810
msgid "Either user or team must be set, not both."
msgstr ""
#: core/models.py:645
#: core/models.py:639
msgid "description"
msgstr ""
#: core/models.py:646
#: core/models.py:640
msgid "code"
msgstr ""
#: core/models.py:647
#: core/models.py:641
msgid "css"
msgstr ""
#: core/models.py:649
#: core/models.py:643
msgid "public"
msgstr ""
#: core/models.py:651
#: core/models.py:645
msgid "Whether this template is public for anyone to use."
msgstr ""
#: core/models.py:657
#: core/models.py:651
msgid "Template"
msgstr ""
#: core/models.py:658
#: core/models.py:652
msgid "Templates"
msgstr ""
#: core/models.py:797
#: core/models.py:791
msgid "Template/user relation"
msgstr ""
#: core/models.py:798
#: core/models.py:792
msgid "Template/user relations"
msgstr ""
#: core/models.py:804
#: core/models.py:798
msgid "This user is already in this template."
msgstr ""
#: core/models.py:810
#: core/models.py:804
msgid "This team is already in this template."
msgstr ""
#: core/models.py:833
#: core/models.py:827
msgid "email address"
msgstr ""
#: core/models.py:850
#: core/models.py:844
msgid "Document invitation"
msgstr ""
#: core/models.py:851
#: core/models.py:845
msgid "Document invitations"
msgstr ""
#: core/models.py:868
#: core/models.py:862
msgid "This email is already associated to a registered user."
msgstr ""
#: core/templates/mail/html/invitation.html:160
#: core/templates/mail/html/invitation2.html:160
#: core/templates/mail/html/hello.html:159 core/templates/mail/text/hello.txt:3
msgid "Company logo"
msgstr ""
#: core/templates/mail/html/hello.html:188 core/templates/mail/text/hello.txt:5
#, python-format
msgid "Hello %(name)s"
msgstr ""
#: core/templates/mail/html/hello.html:188 core/templates/mail/text/hello.txt:5
msgid "Hello"
msgstr ""
#: core/templates/mail/html/hello.html:189 core/templates/mail/text/hello.txt:6
msgid "Thank you very much for your visit!"
msgstr ""
#: core/templates/mail/html/hello.html:221
#, python-format
msgid "This mail has been sent to %(email)s by <a href=\"%(href)s\">%(name)s</a>"
msgstr ""
#: core/templates/mail/html/invitation.html:159
#: core/templates/mail/text/invitation.txt:3
#: core/templates/mail/text/invitation2.txt:3
msgid "La Suite Numérique"
msgstr ""
#: core/templates/mail/html/invitation.html:190
#: core/templates/mail/html/invitation.html:189
#: core/templates/mail/text/invitation.txt:6
#, python-format
msgid " %(username)s shared a document with you ! "
msgstr " %(username)s a partagé un document avec vous ! "
msgid " %(sender_name)s shared a document with you ! "
msgstr " %(sender_name)s a partagé un document avec vous ! "
#: core/templates/mail/html/invitation.html:197
#: core/templates/mail/html/invitation.html:196
#: core/templates/mail/text/invitation.txt:8
#, python-format
msgid " %(username)s invited you as an %(role)s on the following document : "
msgstr " %(username)s vous a invité en tant que %(role)s sur le document suivant : "
msgid " %(sender_name_email)s invited you with the role \"%(role)s\" on the following document : "
msgstr " %(sender_name_email)s vous a invité avec le rôle \"%(role)s\" sur le document suivant : "
#: core/templates/mail/html/invitation.html:206
#: core/templates/mail/html/invitation2.html:211
#: core/templates/mail/html/invitation.html:205
#: core/templates/mail/text/invitation.txt:10
#: core/templates/mail/text/invitation2.txt:11
msgid "Open"
msgstr "Ouvrir"
#: core/templates/mail/html/invitation.html:223
#: core/templates/mail/html/invitation.html:222
#: core/templates/mail/text/invitation.txt:14
msgid " Docs, your new essential tool for organizing, sharing and collaborate on your documents as a team. "
msgid " Docs, your new essential tool for organizing, sharing and collaborating on your documents as a team. "
msgstr " Docs, votre nouvel outil incontournable pour organiser, partager et collaborer sur vos documents en équipe. "
#: core/templates/mail/html/invitation.html:230
#: core/templates/mail/html/invitation2.html:235
#: core/templates/mail/html/invitation.html:229
#: core/templates/mail/text/invitation.txt:16
#: core/templates/mail/text/invitation2.txt:17
msgid "Brought to you by La Suite Numérique"
msgstr "Proposé par La Suite Numérique"
#: core/templates/mail/html/invitation2.html:190
#: core/templates/mail/text/hello.txt:8
#, python-format
msgid "%(username)s shared a document with you"
msgstr "%(username)s a partagé un document avec vous"
#: core/templates/mail/html/invitation2.html:197
#: core/templates/mail/text/invitation2.txt:8
#, python-format
msgid "%(username)s invited you as an %(role)s on the following document :"
msgstr "%(username)s vous a invité en tant que %(role)s sur le document suivant :"
#: core/templates/mail/html/invitation2.html:228
#: core/templates/mail/text/invitation2.txt:15
msgid "Docs, your new essential tool for organizing, sharing and collaborate on your document as a team."
msgstr "Docs, votre nouvel outil essentiel pour organiser, partager et collaborer sur votre document en équipe."
msgid "This mail has been sent to %(email)s by %(name)s [%(href)s]"
msgstr ""
#: impress/settings.py:176
msgid "English"

View File

@@ -25,39 +25,39 @@ license = { file = "LICENSE" }
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
"boto3==1.35.34",
"boto3==1.35.41",
"Brotli==1.1.0",
"celery[redis]==5.4.0",
"django-configurations==2.5.1",
"django-cors-headers==4.4.0",
"django-cors-headers==4.5.0",
"django-countries==7.6.1",
"django-parler==2.3",
"redis==5.1.1",
"django-redis==5.4.0",
"django-storages[s3]==1.14.4",
"django-timezone-field>=5.1",
"django==5.1.1",
"django==5.1.2",
"djangorestframework==3.15.2",
"drf_spectacular==0.27.2",
"dockerflow==2024.4.2",
"easy_thumbnails==2.10",
"factory_boy==3.3.1",
"freezegun==1.5.1",
"gunicorn==23.0.0",
"jsonschema==4.23.0",
"markdown==3.7",
"nested-multipart-parser==1.5.0",
"openai==1.44.1",
"psycopg[binary]==3.2.3",
"PyJWT==2.9.0",
"pypandoc==1.13",
"pypandoc==1.14",
"python-frontmatter==1.1.0",
"python-magic==0.4.27",
"requests==2.32.3",
"sentry-sdk==2.15.0",
"sentry-sdk==2.16.0",
"url-normalize==1.4.3",
"WeasyPrint>=60.2",
"whitenoise==6.7.0",
"mozilla-django-oidc==4.0.1",
"y-py==0.6.2"
]
[project.urls]
@@ -70,10 +70,11 @@ dependencies = [
dev = [
"django-extensions==3.2.3",
"drf-spectacular-sidecar==2024.7.1",
"freezegun==1.5.1",
"ipdb==0.13.13",
"ipython==8.28.0",
"pyfakefs==5.6.0",
"pylint-django==2.5.5",
"pyfakefs==5.7.1",
"pylint-django==2.6.1",
"pylint==3.3.1",
"pytest-cov==5.0.0",
"pytest-django==4.9.0",
@@ -82,7 +83,7 @@ dev = [
"pytest-xdist==3.6.1",
"responses==0.25.3",
"ruff==0.6.9",
"types-requests==2.32.0.20240914",
"types-requests==2.32.0.20241016",
]
[tool.setuptools]

View File

@@ -51,6 +51,7 @@ export const createDoc = async (
await page.locator('.c__modal__backdrop').click({
position: { x: 0, y: 0 },
force: true,
});
await expect(

View File

@@ -181,4 +181,57 @@ test.describe('Doc Editor', () => {
/http:\/\/localhost:8083\/media\/.*\/attachments\/.*.png/,
);
});
test('it checks the AI buttons', async ({ page, browserName }) => {
await page.route(/.*\/ai-translate\//, async (route) => {
const request = route.request();
if (request.method().includes('POST')) {
await route.fulfill({
json: {
answer: 'Bonjour le monde',
},
});
} else {
await route.continue();
}
});
await createDoc(page, 'doc-ai', browserName, 1);
await page.locator('.bn-block-outer').last().fill('Hello World');
const editor = page.locator('.ProseMirror');
await editor.getByText('Hello').dblclick();
await page.getByRole('button', { name: 'AI' }).click();
await expect(
page.getByRole('menuitem', { name: 'Use as prompt' }),
).toBeVisible();
await expect(
page.getByRole('menuitem', { name: 'Rephrase' }),
).toBeVisible();
await expect(
page.getByRole('menuitem', { name: 'Summarize' }),
).toBeVisible();
await expect(page.getByRole('menuitem', { name: 'Correct' })).toBeVisible();
await expect(
page.getByRole('menuitem', { name: 'Language' }),
).toBeVisible();
await page.getByRole('menuitem', { name: 'Language' }).hover();
await expect(
page.getByRole('menuitem', { name: 'English', exact: true }),
).toBeVisible();
await expect(
page.getByRole('menuitem', { name: 'French', exact: true }),
).toBeVisible();
await expect(
page.getByRole('menuitem', { name: 'German', exact: true }),
).toBeVisible();
await page.getByRole('menuitem', { name: 'English', exact: true }).click();
await expect(editor.getByText('Bonjour le monde')).toBeVisible();
});
});

View File

@@ -52,6 +52,7 @@ test.describe('Documents Grid', () => {
orderDefault: '',
orderDesc: '&ordering=-title',
orderAsc: '&ordering=title',
defaultColumn: false,
},
{
nameColumn: 'Created at',
@@ -60,6 +61,7 @@ test.describe('Documents Grid', () => {
orderDefault: '',
orderDesc: '&ordering=-created_at',
orderAsc: '&ordering=created_at',
defaultColumn: false,
},
{
nameColumn: 'Updated at',
@@ -68,6 +70,7 @@ test.describe('Documents Grid', () => {
orderDefault: '&ordering=-updated_at',
orderDesc: '&ordering=updated_at',
orderAsc: '',
defaultColumn: true,
},
].forEach(
({
@@ -77,6 +80,7 @@ test.describe('Documents Grid', () => {
orderDefault,
orderDesc,
orderAsc,
defaultColumn,
}) => {
test(`checks datagrid ordering ${ordering}`, async ({ page }) => {
const responsePromise = page.waitForResponse(
@@ -98,20 +102,19 @@ test.describe('Documents Grid', () => {
);
// Checks the initial state
const datagrid = page
.getByLabel('Datagrid of the documents page 1')
.getByRole('table');
const thead = datagrid.locator('thead');
const datagrid = page.getByLabel('Datagrid of the documents page 1');
const datagridTable = datagrid.getByRole('table');
const thead = datagridTable.locator('thead');
const response = await responsePromise;
expect(response.ok()).toBeTruthy();
const docNameRow1 = datagrid
const docNameRow1 = datagridTable
.getByRole('row')
.nth(1)
.getByRole('cell')
.nth(cellNumber);
const docNameRow2 = datagrid
const docNameRow2 = datagridTable
.getByRole('row')
.nth(2)
.getByRole('cell')
@@ -144,13 +147,21 @@ test.describe('Documents Grid', () => {
await expect(docNameRow2).toHaveText(/.*/);
const textDocNameRow1Asc = await docNameRow1.textContent();
const textDocNameRow2Asc = await docNameRow2.textContent();
const compare = (comp1: string, comp2: string) => {
const comparisonResult = comp1.localeCompare(comp2, 'en', {
caseFirst: 'false',
ignorePunctuation: true,
});
// eslint-disable-next-line playwright/no-conditional-in-test
return defaultColumn ? comparisonResult >= 0 : comparisonResult <= 0;
};
expect(
textDocNameRow1Asc &&
textDocNameRow2Asc &&
textDocNameRow1Asc.localeCompare(textDocNameRow2Asc, 'en', {
caseFirst: 'false',
ignorePunctuation: true,
}) <= 0,
compare(textDocNameRow1Asc, textDocNameRow2Asc),
).toBeTruthy();
// Ordering Desc
@@ -171,10 +182,7 @@ test.describe('Documents Grid', () => {
expect(
textDocNameRow1Desc &&
textDocNameRow2Desc &&
textDocNameRow1Desc.localeCompare(textDocNameRow2Desc, 'en', {
caseFirst: 'false',
ignorePunctuation: true,
}) >= 0,
compare(textDocNameRow2Desc, textDocNameRow1Desc),
).toBeTruthy();
});
},

View File

@@ -384,6 +384,81 @@ test.describe('Doc Header', () => {
}),
).toBeHidden();
});
test('It checks the copy as Markdown button', async ({
page,
browserName,
}) => {
// eslint-disable-next-line playwright/no-skipped-test
test.skip(
browserName === 'webkit',
'navigator.clipboard is not working with webkit and playwright',
);
// create page and navigate to it
await page
.getByRole('button', {
name: 'Create a new document',
})
.click();
// Add dummy content to the doc
const editor = page.locator('.ProseMirror');
const docFirstBlock = editor.locator('.bn-block-content').first();
await docFirstBlock.click();
await page.keyboard.type('# Hello World', { delay: 100 });
const docFirstBlockContent = docFirstBlock.locator('h1');
await expect(docFirstBlockContent).toHaveText('Hello World');
// Copy content to clipboard
await page.getByLabel('Open the document options').click();
await page.getByRole('button', { name: 'Copy as Markdown' }).click();
await expect(page.getByText('Copied to clipboard')).toBeVisible();
// Test that clipboard is in Markdown format
const handle = await page.evaluateHandle(() =>
navigator.clipboard.readText(),
);
const clipboardContent = await handle.jsonValue();
expect(clipboardContent.trim()).toBe('# Hello World');
});
test('It checks the copy as HTML button', async ({ page, browserName }) => {
// eslint-disable-next-line playwright/no-skipped-test
test.skip(
browserName === 'webkit',
'navigator.clipboard is not working with webkit and playwright',
);
// create page and navigate to it
await page
.getByRole('button', {
name: 'Create a new document',
})
.click();
// Add dummy content to the doc
const editor = page.locator('.ProseMirror');
const docFirstBlock = editor.locator('.bn-block-content').first();
await docFirstBlock.click();
await page.keyboard.type('# Hello World', { delay: 100 });
const docFirstBlockContent = docFirstBlock.locator('h1');
await expect(docFirstBlockContent).toHaveText('Hello World');
// Copy content to clipboard
await page.getByLabel('Open the document options').click();
await page.getByRole('button', { name: 'Copy as HTML' }).click();
await expect(page.getByText('Copied to clipboard')).toBeVisible();
// Test that clipboard is in HTML format
const handle = await page.evaluateHandle(() =>
navigator.clipboard.readText(),
);
const clipboardContent = await handle.jsonValue();
expect(clipboardContent.trim()).toBe(
`<h1 data-level="1">Hello World</h1><p></p>`,
);
});
});
test.describe('Documents Header mobile', () => {

View File

@@ -19,17 +19,16 @@ test.describe('Doc Visibility', () => {
const header = page.locator('header').first();
await header.locator('h2').getByText('Docs').click();
const datagrid = page
.getByLabel('Datagrid of the documents page 1')
.getByRole('table');
const datagrid = page.getByLabel('Datagrid of the documents page 1');
const datagridTable = datagrid.getByRole('table');
await expect(datagrid.getByLabel('Loading data')).toBeHidden({
timeout: 10000,
});
await expect(datagrid.getByText(docTitle)).toBeVisible();
await expect(datagridTable.getByText(docTitle)).toBeVisible();
const row = datagrid.getByRole('row').filter({
const row = datagridTable.getByRole('row').filter({
hasText: docTitle,
});

View File

@@ -12,7 +12,7 @@
"test:ui::chromium": "yarn test:ui --project=chromium"
},
"devDependencies": {
"@playwright/test": "1.47.2",
"@playwright/test": "1.48.1",
"@types/node": "*",
"@types/pdf-parse": "1.1.4",
"eslint-config-impress": "*",

View File

@@ -19,36 +19,36 @@
"@blocknote/mantine": "*",
"@blocknote/react": "*",
"@gouvfr-lasuite/integration": "1.0.2",
"@hocuspocus/provider": "2.13.6",
"@hocuspocus/provider": "2.13.7",
"@openfun/cunningham-react": "2.9.4",
"@tanstack/react-query": "5.56.2",
"i18next": "23.15.1",
"@tanstack/react-query": "5.59.15",
"i18next": "23.16.0",
"idb": "8.0.0",
"lodash": "4.17.21",
"luxon": "3.5.0",
"next": "14.2.13",
"next": "14.2.15",
"react": "*",
"react-aria-components": "1.3.3",
"react-aria-components": "1.4.1",
"react-dom": "*",
"react-i18next": "15.0.2",
"react-i18next": "15.0.3",
"react-select": "5.8.1",
"styled-components": "6.1.13",
"y-protocols": "1.0.6",
"yjs": "*",
"zustand": "4.5.5"
"zustand": "5.0.0"
},
"devDependencies": {
"@svgr/webpack": "8.1.0",
"@tanstack/react-query-devtools": "5.58.0",
"@tanstack/react-query-devtools": "5.59.15",
"@testing-library/dom": "10.4.0",
"@testing-library/jest-dom": "6.5.0",
"@testing-library/jest-dom": "6.6.1",
"@testing-library/react": "16.0.1",
"@testing-library/user-event": "14.5.2",
"@types/jest": "29.5.13",
"@types/lodash": "4.17.9",
"@types/lodash": "4.17.10",
"@types/luxon": "3.4.2",
"@types/node": "*",
"@types/react": "18.3.10",
"@types/react": "18.3.11",
"@types/react-dom": "*",
"cross-env": "*",
"dotenv": "16.4.5",
@@ -58,7 +58,7 @@
"jest-environment-jsdom": "29.7.0",
"node-fetch": "2.7.0",
"prettier": "3.3.3",
"stylelint": "16.9.0",
"stylelint": "16.10.0",
"stylelint-config-standard": "36.0.1",
"stylelint-prettier": "5.0.2",
"typescript": "*",

View File

@@ -18,3 +18,7 @@ export class APIError<T = unknown> extends Error implements IAPIError<T> {
this.data = data;
}
}
export const isAPIError = (error: unknown): error is APIError => {
return error instanceof APIError;
};

View File

@@ -0,0 +1,3 @@
export * from './useCreateDocUpload';
export * from './useDocAITransform';
export * from './useDocAITranslate';

View File

@@ -0,0 +1,46 @@
import { useMutation } from '@tanstack/react-query';
import { APIError, errorCauses, fetchAPI } from '@/api';
export type AITransformActions =
| 'correct'
| 'prompt'
| 'rephrase'
| 'summarize';
export type DocAITransform = {
docId: string;
text: string;
action: AITransformActions;
};
export type DocAITransformResponse = {
answer: string;
};
export const docAITransform = async ({
docId,
...params
}: DocAITransform): Promise<DocAITransformResponse> => {
const response = await fetchAPI(`documents/${docId}/ai-transform/`, {
method: 'POST',
body: JSON.stringify({
...params,
}),
});
if (!response.ok) {
throw new APIError(
'Failed to request ai transform',
await errorCauses(response),
);
}
return response.json() as Promise<DocAITransformResponse>;
};
export function useDocAITransform() {
return useMutation<DocAITransformResponse, APIError, DocAITransform>({
mutationFn: docAITransform,
});
}

View File

@@ -0,0 +1,40 @@
import { useMutation } from '@tanstack/react-query';
import { APIError, errorCauses, fetchAPI } from '@/api';
export type DocAITranslate = {
docId: string;
text: string;
language: string;
};
export type DocAITranslateResponse = {
answer: string;
};
export const docAITranslate = async ({
docId,
...params
}: DocAITranslate): Promise<DocAITranslateResponse> => {
const response = await fetchAPI(`documents/${docId}/ai-translate/`, {
method: 'POST',
body: JSON.stringify({
...params,
}),
});
if (!response.ok) {
throw new APIError(
'Failed to request ai translate',
await errorCauses(response),
);
}
return response.json() as Promise<DocAITranslateResponse>;
};
export function useDocAITranslate() {
return useMutation<DocAITranslateResponse, APIError, DocAITranslate>({
mutationFn: docAITranslate,
});
}

View File

@@ -0,0 +1,384 @@
import {
ComponentProps,
useBlockNoteEditor,
useComponentsContext,
useSelectedBlocks,
} from '@blocknote/react';
import {
Loader,
VariantType,
useToastProvider,
} from '@openfun/cunningham-react';
import {
PropsWithChildren,
ReactNode,
useCallback,
useEffect,
useMemo,
useState,
} from 'react';
import { useTranslation } from 'react-i18next';
import { isAPIError } from '@/api';
import { Box, Text } from '@/components';
import { useDocOptions } from '@/features/docs/doc-management/';
import {
AITransformActions,
useDocAITransform,
useDocAITranslate,
} from '../api/';
import { useDocStore } from '../stores';
type LanguageTranslate = {
value: string;
display_name: string;
};
const sortByPopularLanguages = (
languages: LanguageTranslate[],
popularLanguages: string[],
) => {
languages.sort((a, b) => {
const indexA = popularLanguages.indexOf(a.value);
const indexB = popularLanguages.indexOf(b.value);
// If both languages are in the popular list, sort based on their order in popularLanguages
if (indexA !== -1 && indexB !== -1) {
return indexA - indexB;
}
// If only a is in the popular list, it should come first
if (indexA !== -1) {
return -1;
}
// If only b is in the popular list, it should come first
if (indexB !== -1) {
return 1;
}
// If neither a nor b is in the popular list, maintain their relative order
return 0;
});
};
export function AIGroupButton() {
const editor = useBlockNoteEditor();
const Components = useComponentsContext();
const selectedBlocks = useSelectedBlocks(editor);
const { t } = useTranslation();
const { currentDoc } = useDocStore();
const { data: docOptions } = useDocOptions();
const [languages, setLanguages] = useState<LanguageTranslate[]>([]);
useEffect(() => {
const languages = docOptions?.actions.POST.language.choices;
if (!languages) {
return;
}
sortByPopularLanguages(languages, [
'fr',
'en',
'de',
'es',
'it',
'pt',
'nl',
'pl',
]);
setLanguages(languages);
}, [docOptions?.actions.POST.language.choices]);
const show = useMemo(() => {
return !!selectedBlocks.find((block) => block.content !== undefined);
}, [selectedBlocks]);
if (!show || !editor.isEditable || !Components || !currentDoc || !languages) {
return null;
}
return (
<Components.Generic.Menu.Root>
<Components.Generic.Menu.Trigger>
<Components.FormattingToolbar.Button
className="bn-button bn-menu-item"
data-test="ai-actions"
label="AI"
mainTooltip={t('AI Actions')}
icon={
<Text $isMaterialIcon $size="l">
auto_awesome
</Text>
}
/>
</Components.Generic.Menu.Trigger>
<Components.Generic.Menu.Dropdown
className="bn-menu-dropdown bn-drag-handle-menu"
sub={true}
>
<AIMenuItemTransform
action="prompt"
docId={currentDoc.id}
icon={
<Text $isMaterialIcon $size="s">
text_fields
</Text>
}
>
{t('Use as prompt')}
</AIMenuItemTransform>
<AIMenuItemTransform
action="rephrase"
docId={currentDoc.id}
icon={
<Text $isMaterialIcon $size="s">
refresh
</Text>
}
>
{t('Rephrase')}
</AIMenuItemTransform>
<AIMenuItemTransform
action="summarize"
docId={currentDoc.id}
icon={
<Text $isMaterialIcon $size="s">
summarize
</Text>
}
>
{t('Summarize')}
</AIMenuItemTransform>
<AIMenuItemTransform
action="correct"
docId={currentDoc.id}
icon={
<Text $isMaterialIcon $size="s">
check
</Text>
}
>
{t('Correct')}
</AIMenuItemTransform>
<Components.Generic.Menu.Root position="right" sub={true}>
<Components.Generic.Menu.Trigger sub={false}>
<Components.Generic.Menu.Item
className="bn-menu-item"
subTrigger={true}
>
<Box $direction="row" $gap="0.6rem">
<Text $isMaterialIcon $size="s">
translate
</Text>
{t('Language')}
</Box>
</Components.Generic.Menu.Item>
</Components.Generic.Menu.Trigger>
<Components.Generic.Menu.Dropdown
sub={true}
className="bn-menu-dropdown"
>
{languages.map((language) => (
<AIMenuItemTranslate
key={language.value}
language={language.value}
docId={currentDoc.id}
>
{language.display_name}
</AIMenuItemTranslate>
))}
</Components.Generic.Menu.Dropdown>
</Components.Generic.Menu.Root>
</Components.Generic.Menu.Dropdown>
</Components.Generic.Menu.Root>
);
}
/**
* Item is derived from Mantime, some props seem lacking or incorrect.
*/
type ItemDefault = ComponentProps['Generic']['Menu']['Item'];
type ItemProps = Omit<ItemDefault, 'onClick'> & {
rightSection?: ReactNode;
closeMenuOnClick?: boolean;
onClick: (e: React.MouseEvent) => void;
};
interface AIMenuItemTransform {
action: AITransformActions;
docId: string;
icon?: ReactNode;
}
const AIMenuItemTransform = ({
docId,
action,
children,
icon,
}: PropsWithChildren<AIMenuItemTransform>) => {
const editor = useBlockNoteEditor();
const { mutateAsync: requestAI, isPending } = useDocAITransform();
const handleAIError = useHandleAIError();
const handleAIAction = useCallback(async () => {
const selectedBlocks = editor.getSelection()?.blocks;
if (!selectedBlocks || selectedBlocks.length === 0) {
return;
}
const markdown = await editor.blocksToMarkdownLossy(selectedBlocks);
try {
const responseAI = await requestAI({
text: markdown,
action,
docId,
});
if (!responseAI.answer) {
return;
}
const blockMarkdown = await editor.tryParseMarkdownToBlocks(
responseAI.answer,
);
editor.replaceBlocks(selectedBlocks, blockMarkdown);
} catch (error) {
handleAIError(error);
}
}, [editor, requestAI, action, docId, handleAIError]);
return (
<AIMenuItem
icon={icon}
handleAIAction={handleAIAction}
isPending={isPending}
>
{children}
</AIMenuItem>
);
};
interface AIMenuItemTranslate {
language: string;
docId: string;
icon?: ReactNode;
}
const AIMenuItemTranslate = ({
children,
docId,
icon,
language,
}: PropsWithChildren<AIMenuItemTranslate>) => {
const editor = useBlockNoteEditor();
const { mutateAsync: requestAI, isPending } = useDocAITranslate();
const handleAIError = useHandleAIError();
const handleAIAction = useCallback(async () => {
const selectedBlocks = editor.getSelection()?.blocks;
if (!selectedBlocks || selectedBlocks.length === 0) {
return;
}
const markdown = await editor.blocksToMarkdownLossy(selectedBlocks);
try {
const responseAI = await requestAI({
text: markdown,
language,
docId,
});
if (!responseAI.answer) {
return;
}
const blockMarkdown = await editor.tryParseMarkdownToBlocks(
responseAI.answer,
);
editor.replaceBlocks(selectedBlocks, blockMarkdown);
} catch (error) {
handleAIError(error);
}
}, [editor, requestAI, language, docId, handleAIError]);
return (
<AIMenuItem
icon={icon}
handleAIAction={handleAIAction}
isPending={isPending}
>
{children}
</AIMenuItem>
);
};
interface AIMenuItemProps {
handleAIAction: () => Promise<void>;
isPending: boolean;
icon?: ReactNode;
}
const AIMenuItem = ({
handleAIAction,
isPending,
children,
icon,
}: PropsWithChildren<AIMenuItemProps>) => {
const Components = useComponentsContext();
if (!Components) {
return null;
}
const Item = Components.Generic.Menu.Item as React.FC<ItemProps>;
return (
<Item
closeMenuOnClick={false}
icon={icon}
onClick={(e: React.MouseEvent) => {
e.stopPropagation();
void handleAIAction();
}}
rightSection={isPending ? <Loader size="small" /> : undefined}
>
{children}
</Item>
);
};
const useHandleAIError = () => {
const { toast } = useToastProvider();
const { t } = useTranslation();
const handleAIError = useCallback(
(error: unknown) => {
if (isAPIError(error)) {
error.cause?.forEach((cause) => {
if (
cause === 'Request was throttled. Expected available in 60 seconds.'
) {
toast(
t('Too many requests. Please wait 60 seconds.'),
VariantType.ERROR,
);
}
});
}
toast(t('AI seems busy! Please try again.'), VariantType.ERROR);
console.error(error);
},
[toast, t],
);
return handleAIError;
};

View File

@@ -12,6 +12,7 @@ import {
} from '@blocknote/react';
import React from 'react';
import { AIGroupButton } from './AIButton';
import { MarkdownButton } from './MarkdownButton';
export const BlockNoteToolbar = () => {
@@ -21,6 +22,9 @@ export const BlockNoteToolbar = () => {
<FormattingToolbar>
<BlockTypeSelect key="blockTypeSelect" />
{/* Extra button to do some AI powered actions */}
<AIGroupButton key="AIButton" />
{/* Extra button to convert from markdown to json */}
<MarkdownButton key="customButton" />

View File

@@ -4,7 +4,7 @@ import * as Y from 'yjs';
import { create } from 'zustand';
import { providerUrl } from '@/core';
import { Base64 } from '@/features/docs/doc-management';
import { Base64, Doc } from '@/features/docs/doc-management';
import { blocksToYDoc } from '../utils';
@@ -14,14 +14,17 @@ interface DocStore {
}
export interface UseDocStore {
currentDoc?: Doc;
docsStore: {
[storeId: string]: DocStore;
};
createProvider: (storeId: string, initialDoc: Base64) => HocuspocusProvider;
setStore: (storeId: string, props: Partial<DocStore>) => void;
setCurrentDoc: (doc: Doc | undefined) => void;
}
export const useDocStore = create<UseDocStore>((set, get) => ({
currentDoc: undefined,
docsStore: {},
createProvider: (storeId: string, initialDoc: Base64) => {
const doc = new Y.Doc({
@@ -52,8 +55,9 @@ export const useDocStore = create<UseDocStore>((set, get) => ({
return provider;
},
setStore: (storeId, props) => {
set(({ docsStore }) => {
set(({ docsStore }, ...store) => {
return {
...store,
docsStore: {
...docsStore,
[storeId]: {
@@ -64,4 +68,7 @@ export const useDocStore = create<UseDocStore>((set, get) => ({
};
});
},
setCurrentDoc: (doc) => {
set({ currentDoc: doc });
},
}));

View File

@@ -1,10 +1,14 @@
import { Button } from '@openfun/cunningham-react';
import {
Button,
VariantType,
useToastProvider,
} from '@openfun/cunningham-react';
import React, { useState } from 'react';
import { useTranslation } from 'react-i18next';
import { Box, DropButton, IconOptions } from '@/components';
import { useAuthStore } from '@/core';
import { usePanelEditorStore } from '@/features/docs/doc-editor/';
import { useDocStore, usePanelEditorStore } from '@/features/docs/doc-editor/';
import {
Doc,
ModalRemoveDoc,
@@ -31,6 +35,32 @@ export const DocToolBox = ({ doc, versionId }: DocToolBoxProps) => {
const [isModalVersionOpen, setIsModalVersionOpen] = useState(false);
const { isSmallMobile } = useResponsiveStore();
const { authenticated } = useAuthStore();
const { docsStore } = useDocStore();
const { toast } = useToastProvider();
const copyCurrentEditorToClipboard = async (
asFormat: 'html' | 'markdown',
) => {
const editor = docsStore[doc.id]?.editor;
if (!editor) {
toast(t('Editor unavailable'), VariantType.ERROR, { duration: 3000 });
return;
}
try {
const editorContentFormatted =
asFormat === 'html'
? await editor.blocksToHTMLLossy()
: await editor.blocksToMarkdownLossy();
await navigator.clipboard.writeText(editorContentFormatted);
toast(t('Copied to clipboard'), VariantType.SUCCESS, { duration: 3000 });
} catch (error) {
console.error(error);
toast(t('Failed to copy to clipboard'), VariantType.ERROR, {
duration: 3000,
});
}
};
return (
<Box
@@ -125,6 +155,28 @@ export const DocToolBox = ({ doc, versionId }: DocToolBoxProps) => {
{t('Delete document')}
</Button>
)}
<Button
onClick={() => {
setIsDropOpen(false);
void copyCurrentEditorToClipboard('markdown');
}}
color="primary-text"
icon={<span className="material-icons">content_copy</span>}
size="small"
>
{t('Copy as {{format}}', { format: 'Markdown' })}
</Button>
<Button
onClick={() => {
setIsDropOpen(false);
void copyCurrentEditorToClipboard('html');
}}
color="primary-text"
icon={<span className="material-icons">content_copy</span>}
size="small"
>
{t('Copy as {{format}}', { format: 'HTML' })}
</Button>
</Box>
</DropButton>
</Box>

View File

@@ -1,5 +1,6 @@
export * from './useCreateDoc';
export * from './useDoc';
export * from './useDocOptions';
export * from './useDocs';
export * from './useUpdateDoc';
export * from './useUpdateDocLink';

View File

@@ -0,0 +1,47 @@
import { UseQueryOptions, useQuery } from '@tanstack/react-query';
import { APIError, errorCauses, fetchAPI } from '@/api';
type DocOptionsResponse = {
actions: {
POST: {
language: {
choices: {
value: string;
display_name: string;
}[];
};
};
};
};
export const docOptions = async (): Promise<DocOptionsResponse> => {
const response = await fetchAPI(`documents/`, {
method: 'OPTIONS',
});
if (!response.ok) {
throw new APIError(
'Failed to get the doc options',
await errorCauses(response),
);
}
return response.json() as Promise<DocOptionsResponse>;
};
export const KEY_DOC_OPTIONS = 'doc-options';
export function useDocOptions(
queryConfig?: UseQueryOptions<
DocOptionsResponse,
APIError,
DocOptionsResponse
>,
) {
return useQuery<DocOptionsResponse, APIError, DocOptionsResponse>({
queryKey: [KEY_DOC_OPTIONS],
queryFn: docOptions,
...queryConfig,
});
}

View File

@@ -3,11 +3,14 @@
"fr": {
"translation": {
"\"{{email}}\" is already invited to the document.": "\"{{email}}\" est déjà invité à accéder au document.",
"\"{{email}}\" is already member of the document.": "\"{{email}}\" est déjà membre du document.",
"AI Actions": "Actions IA",
"AI seems busy! Please try again.": "L'IA semble occupée ! Veuillez réessayer.",
"Accessibility": "Accessibilité",
"Accessibility statement": "Déclaration d'accessibilité",
"Address:": "Adresse :",
"Administrator": "Administrateur",
"Anyone on the internet with the link can view": "N'importe qui sur Internet avec le lien peut consulter",
"Anyone on the internet with the link can view": "Les personnes disposant du lien peuvent y accéder",
"Are you sure you want to delete the document \"{{title}}\"?": "Êtes-vous sûr de vouloir supprimer le document \"{{title}}\" ?",
"Back to home page": "Retour à l'accueil",
"Back to top": "Retour en haut",
@@ -21,8 +24,11 @@
"Content modal to delete document": "Contenu modal pour supprimer le document",
"Content modal to export the document": "Contenu modal pour exporter le document",
"Cookies placed": "Cookies déposés",
"Copied to clipboard": "Copié dans le presse-papier",
"Copy as {{format}}": "Copier en {{format}}",
"Copy link": "Copier le lien",
"Copyright": "Copyright",
"Correct": "Corriger",
"Create a new document": "Créer un nouveau document",
"Created at": "Créé le",
"Current version": "Version en cours",
@@ -44,11 +50,13 @@
"Download": "Télécharger",
"E-mail:": "E-mail:",
"Editor": "Éditeur",
"Editor unavailable": "Éditeur indisponible",
"Established on December 20, 2023.": "Établi le 20 décembre 2023.",
"Export": "Exporter",
"Export your document, it will be inserted in the selected template.": "Exportez votre document, il sera inséré dans le modèle sélectionné.",
"Failed to add the member in the document.": "Impossible d'ajouter le membre dans le document.",
"Failed to copy link": "Échec de la copie du lien",
"Failed to copy to clipboard": "Échec de la copie dans le presse-papier",
"Failed to create the invitation for {{email}}.": "Impossible de créer l'invitation pour {{email}}.",
"Find a member to add to the document": "Trouver un membre à ajouter au document",
"French Interministerial Directorate for Digital Affairs (DINUM), 20 avenue de Ségur 75007 Paris.": "Direction interministérielle des affaires numériques (DINUM), 20 avenue de Segur 75007 Paris.",
@@ -64,7 +72,6 @@
"It seems that the page you are looking for does not exist or cannot be displayed correctly.": "Il semble que la page que vous cherchez n'existe pas ou ne puisse pas être affichée correctement.",
"It's true, you didn't have to click on a block that covers half the page to say you agree to the placement of cookies — even if you don't know what it means!": "C'est vrai, vous n'avez pas à cliquer sur un bloc qui couvre la moitié de la page pour dire que vous acceptez le placement de cookies — même si vous ne savez pas ce que cela signifie !",
"Language": "Langue",
"Language Icon": "Icône de langue",
"Legal Notice": "Mentions Legales",
"Legal notice": "Mention légale",
"Link Copied !": "Lien copié !",
@@ -80,6 +87,7 @@
"Offline ?!": "Hors-ligne ?!",
"Only for people with access": "Seulement pour les personnes avec accès",
"Open the document options": "Ouvrir les options du document",
"Open the header menu": "Ouvrir le menu d'en-tête",
"Open the panel": "Ouvrir le panneau",
"Open the version options": "Ouvrir les options de version",
"Ouch !": "Aïe !",
@@ -95,6 +103,7 @@
"Reader": "Lecteur",
"Remedies": "Voie de recours",
"Rename": "Renommer",
"Rephrase": "Reformuler",
"Restore": "Restaurer",
"Restore the version": "Restaurer la version",
"Restore this version": "Restaurer cette version",
@@ -106,6 +115,7 @@
"Share modal": "Modale de partage",
"Something bad happens, please retry.": "Une erreur inattendue s'est produite, veuillez réessayer.",
"Stéphanie Schaer: Interministerial Digital Director (DINUM).": "Stéphanie Schaer: Directrice numérique interministériel (DINUM).",
"Summarize": "Résumer",
"Table of content": "Table des matières",
"Table of contents": "Table des matières",
"Template": "Template",
@@ -122,9 +132,11 @@
"This site does not display a cookie consent banner, why?": "Ce site n'affiche pas de bannière de consentement des cookies, pourquoi?",
"This site places a small text file (a \"cookie\") on your computer when you visit it.": "Ce site place un petit fichier texte (un « cookie ») sur votre ordinateur lorsque vous le visitez.",
"This will protect your privacy, but will also prevent the owner from learning from your actions and creating a better experience for you and other users.": "Cela protégera votre vie privée, mais empêchera également le propriétaire d'apprendre de vos actions et de créer une meilleure expérience pour vous et les autres utilisateurs.",
"Too many requests. Please wait 60 seconds.": "Trop de demandes. Veuillez patienter 60 secondes.",
"Unless otherwise stated, all content on this site is under": "Sauf mention contraire, tout le contenu de ce site est sous",
"Untitled document": "Document sans titre",
"Updated at": "Mise à jour le",
"Use as prompt": "Utiliser comme un prompt",
"User {{email}} added to the document.": "L'utilisateur {{email}} a été ajouté au document.",
"Validate": "Valider",
"Version history": "Historique des versions",

View File

@@ -6,7 +6,7 @@ import { useEffect, useState } from 'react';
import { Box, Text } from '@/components';
import { TextErrors } from '@/components/TextErrors';
import { useAuthStore } from '@/core/auth';
import { DocEditor } from '@/features/docs';
import { DocEditor, useDocStore } from '@/features/docs';
import { useDoc } from '@/features/docs/doc-management';
import { MainLayout } from '@/layouts';
import { NextPageWithLayout } from '@/types/next';
@@ -35,6 +35,7 @@ const DocPage = ({ id }: DocProps) => {
const { login } = useAuthStore();
const { data: docQuery, isError, error } = useDoc({ id });
const [doc, setDoc] = useState(docQuery);
const { setCurrentDoc } = useDocStore();
const navigate = useNavigate();
@@ -52,7 +53,12 @@ const DocPage = ({ id }: DocProps) => {
}
setDoc(docQuery);
}, [docQuery]);
setCurrentDoc(docQuery);
return () => {
setCurrentDoc(undefined);
};
}, [docQuery, setCurrentDoc]);
if (isError && error) {
if (error.status === 404) {

View File

@@ -28,15 +28,15 @@
"@blocknote/core": "0.16.0",
"@blocknote/mantine": "0.16.0",
"@blocknote/react": "0.16.0",
"@types/node": "20.16.10",
"@types/react-dom": "18.3.0",
"@typescript-eslint/eslint-plugin": "8.7.0",
"@typescript-eslint/parser": "8.7.0",
"@types/node": "20.16.12",
"@types/react-dom": "18.3.1",
"@typescript-eslint/eslint-plugin": "8.9.0",
"@typescript-eslint/parser": "8.9.0",
"cross-env": "7.0.3",
"eslint": "8.57.0",
"react": "18.3.1",
"react-dom": "18.3.1",
"typescript": "5.6.2",
"yjs": "13.6.19"
"typescript": "5.6.3",
"yjs": "13.6.20"
}
}

View File

@@ -6,19 +6,19 @@
"lint": "eslint --ext .js ."
},
"dependencies": {
"@next/eslint-plugin-next": "14.2.13",
"@tanstack/eslint-plugin-query": "5.58.1",
"@next/eslint-plugin-next": "14.2.15",
"@tanstack/eslint-plugin-query": "5.59.7",
"@typescript-eslint/eslint-plugin": "*",
"@typescript-eslint/parser": "*",
"eslint": "*",
"eslint-config-next": "14.2.13",
"eslint-config-next": "14.2.15",
"eslint-config-prettier": "9.1.0",
"eslint-plugin-import": "2.30.0",
"eslint-plugin-import": "2.31.0",
"eslint-plugin-jest": "28.8.3",
"eslint-plugin-jsx-a11y": "6.10.0",
"eslint-plugin-playwright": "1.6.2",
"eslint-plugin-playwright": "1.7.0",
"eslint-plugin-prettier": "5.2.1",
"eslint-plugin-react": "7.37.0",
"eslint-plugin-react": "7.37.1",
"eslint-plugin-testing-library": "6.3.0",
"prettier": "3.3.3"
}

View File

@@ -15,7 +15,7 @@
"@types/jest": "29.5.13",
"@types/node": "*",
"eslint-config-impress": "*",
"eslint-plugin-import": "2.30.0",
"eslint-plugin-import": "2.31.0",
"i18next-parser": "9.0.2",
"jest": "29.7.0",
"ts-jest": "29.2.5",

View File

@@ -15,7 +15,7 @@
"node": ">=18"
},
"dependencies": {
"@hocuspocus/server": "2.13.6",
"@hocuspocus/server": "2.13.7",
"y-protocols": "1.0.6"
},
"devDependencies": {

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,67 @@
aiBaseUrl: ENC[AES256_GCM,data:HKUEyJUP94wrZU6p6ezRc2c+hDBj2m2/Jw==,iv:qMka2VprAKjU8Q8F5+mtm6MY5cPWHdIBDjADiPHR1iQ=,tag:ykOJhJ07gFHALF7ZpAp5ig==,type:str]
aiApiKey: ENC[AES256_GCM,data:zY59S7I7DnPXlb6OcwFNsJdocXTJKVtcfvPl1nkgC9o=,iv:go72+zykuJpPLiLxkrLwP6lVjnGCnzBMoXLtnPivvuQ=,tag:cvV1UIHx/IrbftwsUGsviQ==,type:str]
sops:
kms: []
gcp_kms: []
azure_kv: []
hc_vault: []
age:
- recipient: age15fyxdwmg5mvldtqqus87xspuws2u0cpvwheehrtvkexj4tnsqqysw6re2x
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBwNDl5OW83MUwwejVCSE10
SkV3SGZHZWpONnZqYWlCUmdJZmppM2d5YmxFCnRvTExZMnlwZTJUYkhXcWVSMnNr
QjBubVhYbkxFVUlQUFRNcHRiWGhueGsKLS0tIFBrYUhQNVhpQ2wwZjhObzVzME5E
QmtMOC9waTZTekc1alpBdXRHbWpNME0KTtfTufsr+kZ0/y3gaZjU+lT8QAIakzoh
rCojnZgi7chIJPwFRbNeDizVPtvawET+pEYBUGpEXVLTOVt91rWhKQ==
-----END AGE ENCRYPTED FILE-----
- recipient: age16hnlml8yv4ynwy0seer57g8qww075crd0g7nsundz3pj4wk7m3vqftszg7
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBSNFRrem9JV0diME5QMW9M
T2dMVExBYWVPYzY4ck5aNzJBN1FYc1ZMM1g4CmM3UkUxLzN2dnlhUVFsUXRQY0tL
aW9NSlQ1aG9mb2MxT3ZRRTZjNUMyVWsKLS0tIElXVFFmdEtEZnJ0eVRFR21heE5W
dU5RSkd4ZG9qa2U3MGM3VjFqMERNRUkK3yJXuOTlIv+X+vb07olarV+RfCcxVJ7e
WOPfC7S3RfRaf5Ic//rGHeaO7NSV9qFRIeyhM+DP3HclCI4nh1A14A==
-----END AGE ENCRYPTED FILE-----
- recipient: age1qy04neuzwpasmvljqrcvhwnf0kz5cpyteze38c8avp0czewskasszv9pyw
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBrZWp6SzV5N2MxbVdRcEd4
WmN3d1Frc1MweUt4M21tdFZmRXRJUTRQREV3Cm1RM2lyYUVYTjdmR29nT0piS3dR
OTQxdU80NmF4eG45VE1uQ1VXb29ubU0KLS0tIFNxb0RyRXJWV0xheDZLa1NTNWtC
bTBMWkh1VEF3bW9wdm5vTTRmR1NYcHMKsPhh5zKRBKYnabQfK4x85hJ56nTM/b1t
PAvBgYYwYRwVdv7UP6FsNnU+fIxV1g19PF7iLZlZVwfuNgIASPJtAg==
-----END AGE ENCRYPTED FILE-----
- recipient: age1plkp8td6zzfcavjusmsfrlk54t9vn8jjxm8zaz7cmnr7kzl2nfnsd54hwg
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBDNmh3TC9IYmlnYlBEZHVB
YitRNEZucWhaY0xJdEdsZWlaKzhSYTFYY1dFClVqdFM4TmpsWC9YQ3RUcEpnVURH
NjV0QU1COG4vQkhGeWdyeUhjUWpyZDAKLS0tIHJnOWVIZmFGRHNLWXVnSUFieFBt
d1R6d0hLZloxRmtlczd1UGtaTERqYW8KqTie+Oq9dqzdBoaLueL/OCEvwHfMiSpZ
wPJkOhJYLQAf+0WfO1CxrWOMmTPyXwR8vNXEnhitBwrg8s/h1fYEhw==
-----END AGE ENCRYPTED FILE-----
- recipient: age12g6f5fse25tgrwweleh4jls3qs52hey2edh759smulwmk5lnzadslu2cp3
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBKSzNqZS93YUQ4MmVoUGF5
OXFxVlppalVIeGtsemM0Q0gzVXRmQUw4YUV3CkVBNFhkVWRTdGZsTnVnTGQySUc3
dHNxNlJFOVZjcVZ3ckNlU3hxMXRYbGMKLS0tIDVuMlVNdUNNU09jc2dPY2RPamt2
Zk9DenZpUEx4QXMxZEZsVHVaRGFqSDAKw2rmGTV5iWXApdqBRaNLFYBc7qYadLGc
NRztmZNOGgG9N0P+Zv+1IEXotLJ/8CnCpyYaV3JbAYmGGZFYZTBLQg==
-----END AGE ENCRYPTED FILE-----
- recipient: age1hnhuzj96ktkhpyygvmz0x9h8mfvssz7ss6emmukags644mdhf4msajk93r
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBtTVBMb1NjNkRla3BEMjc0
dVNxTTJTVGxudW9hMVVIYXh3aGVIOUVWR0EwCk0weVltQStJb1cxYWlTdStHZUgx
eEI1R2xqSGNzOTJUeGJLK2NOVGhPVnMKLS0tIHhLUFJvVHBWeWdxTXJWS1NScGZq
WWpmQmwvSjR3dlNZQy80MEhNVVZtdUkKCGDakeuRxdIgFwVS8D9mBT6VUUp1JTLW
t18K0eHuNQW4M9ZrxSrrTHQkQ60e3/GJym7OAkhnwA7dt+G9hg6Jbw==
-----END AGE ENCRYPTED FILE-----
lastmodified: "2024-10-18T09:35:17Z"
mac: ENC[AES256_GCM,data:Rk8HeFcfsSPEZdfM6LYGrZgMl6uev+0nrLApyqtke1gXUj5cAGymNmKAmRUuE3IO5iHFn13c67FPDYeK28FSgPZ6r+ae/IE+ypCT6UnLx/HznLUdmo6dI81SNeUlCGPwgVe2V+/LThknF85MSo5fQks2/OzoRinDBhGfgt8e2so=,iv:V/nrhZnxfMr2uJbdza6ATHmZN41F982jO7UhjOjBh30=,tag:avXISYuDAvYywbFxfgnwGQ==,type:str]
pgp: []
unencrypted_suffix: _unencrypted
version: 3.9.0

View File

@@ -0,0 +1 @@
../../../../secrets/numerique-gouv/impress/env/docs-ia/secrets.enc.yaml

View File

@@ -0,0 +1,186 @@
image:
repository: lasuite/impress-backend
pullPolicy: Always
tag: "vdemo"
backend:
migrateJobAnnotations:
argocd.argoproj.io/hook: PreSync
argocd.argoproj.io/hook-delete-policy: HookSucceeded
envVars:
AI_API_KEY:
secretKeyRef:
name: backend
key: AI_API_KEY
AI_BASE_URL:
secretKeyRef:
name: backend
key: AI_BASE_URL
AI_DOCUMENT_RATE_THROTTLE_RATES: '{"minute": 5000, "hour": 5000, "day": 5000}'
AI_MODEL: ministral-8b-latest
AI_USER_RATE_THROTTLE_RATES: '{"minute": 5000, "hour": 5000, "day": 5000}'
DJANGO_CSRF_TRUSTED_ORIGINS: http://docs-ia.beta.numerique.gouv.fr,https://docs-ia.beta.numerique.gouv.fr
DJANGO_CONFIGURATION: Production
DJANGO_ALLOWED_HOSTS: "*"
DJANGO_SECRET_KEY:
secretKeyRef:
name: backend
key: DJANGO_SECRET_KEY
DJANGO_SETTINGS_MODULE: impress.settings
DJANGO_SUPERUSER_EMAIL:
secretKeyRef:
name: backend
key: DJANGO_SUPERUSER_EMAIL
DJANGO_SUPERUSER_PASSWORD:
secretKeyRef:
name: backend
key: DJANGO_SUPERUSER_PASSWORD
DJANGO_EMAIL_HOST: "smtp.tem.scw.cloud"
DJANGO_EMAIL_PORT: 587
DJANGO_EMAIL_USE_TLS: True
DJANGO_EMAIL_FROM: "noreply@docs.beta.numerique.gouv.fr"
DJANGO_EMAIL_HOST_USER:
secretKeyRef:
name: backend
key: DJANGO_EMAIL_HOST_USER
DJANGO_EMAIL_HOST_PASSWORD:
secretKeyRef:
name: backend
key: DJANGO_EMAIL_HOST_PASSWORD
DJANGO_SILENCED_SYSTEM_CHECKS: security.W008,security.W004
OIDC_OP_JWKS_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/jwks
OIDC_OP_AUTHORIZATION_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/authorize
OIDC_OP_TOKEN_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/token
OIDC_OP_USER_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/userinfo
OIDC_OP_LOGOUT_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/session/end
OIDC_RP_CLIENT_ID:
secretKeyRef:
name: backend
key: OIDC_RP_CLIENT_ID
OIDC_RP_CLIENT_SECRET:
secretKeyRef:
name: backend
key: OIDC_RP_CLIENT_SECRET
OIDC_RP_SIGN_ALGO: RS256
OIDC_RP_SCOPES: "openid email"
OIDC_REDIRECT_ALLOWED_HOSTS: https://docs-ia.beta.numerique.gouv.fr
OIDC_AUTH_REQUEST_EXTRA_PARAMS: "{'acr_values': 'eidas1'}"
LOGIN_REDIRECT_URL: https://docs-ia.beta.numerique.gouv.fr
LOGIN_REDIRECT_URL_FAILURE: https://docs-ia.beta.numerique.gouv.fr
LOGOUT_REDIRECT_URL: https://docs-ia.beta.numerique.gouv.fr
DB_HOST:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: host
DB_NAME:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: database
DB_USER:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: username
DB_PASSWORD:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: password
DB_PORT:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: port
POSTGRES_USER:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: username
POSTGRES_DB:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: database
POSTGRES_PASSWORD:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: password
REDIS_URL:
secretKeyRef:
name: redis.redis.libre.sh
key: url
AWS_S3_ENDPOINT_URL:
secretKeyRef:
name: impress-media-storage.bucket.libre.sh
key: url
AWS_S3_ACCESS_KEY_ID:
secretKeyRef:
name: impress-media-storage.bucket.libre.sh
key: accessKey
AWS_S3_SECRET_ACCESS_KEY:
secretKeyRef:
name: impress-media-storage.bucket.libre.sh
key: secretKey
AWS_STORAGE_BUCKET_NAME:
secretKeyRef:
name: impress-media-storage.bucket.libre.sh
key: bucket
AWS_S3_REGION_NAME: local
STORAGES_STATICFILES_BACKEND: django.contrib.staticfiles.storage.StaticFilesStorage
createsuperuser:
command:
- "/bin/sh"
- "-c"
- |
python manage.py createsuperuser --email $DJANGO_SUPERUSER_EMAIL --password $DJANGO_SUPERUSER_PASSWORD
restartPolicy: Never
frontend:
image:
repository: lasuite/impress-frontend
pullPolicy: Always
tag: "vdemo"
yProvider:
image:
repository: lasuite/impress-y-provider
pullPolicy: Always
tag: "vdemo"
ingress:
enabled: true
host: docs-ia.beta.numerique.gouv.fr
className: nginx
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
ingressWS:
enabled: true
host: docs-ia.beta.numerique.gouv.fr
className: nginx
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
ingressAdmin:
enabled: true
host: docs-ia.beta.numerique.gouv.fr
className: nginx
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/auth-signin: https://oauth2-proxy-preprod.beta.numerique.gouv.fr/oauth2/start
nginx.ingress.kubernetes.io/auth-url: https://oauth2-proxy-preprod.beta.numerique.gouv.fr/oauth2/auth
ingressMedia:
enabled: true
host: docs-ia.beta.numerique.gouv.fr
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/auth-response-headers: "Authorization, X-Amz-Date, X-Amz-Content-SHA256"
nginx.ingress.kubernetes.io/auth-url: https://docs-ia.beta.numerique.gouv.fr/api/v1.0/documents/retrieve-auth/
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/rewrite-target: /docs-ia-impress-media-storage/$1
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/upstream-vhost: s3.margaret-hamilton.indiehosters.net
serviceMedia:
host: s3.margaret-hamilton.indiehosters.net
port: 443

View File

@@ -8,6 +8,15 @@ backend:
argocd.argoproj.io/hook: PreSync
argocd.argoproj.io/hook-delete-policy: HookSucceeded
envVars:
AI_API_KEY:
secretKeyRef:
name: backend
key: AI_API_KEY
AI_BASE_URL:
secretKeyRef:
name: backend
key: AI_BASE_URL
AI_MODEL: meta-llama/Meta-Llama-3.1-70B-Instruct
DJANGO_CSRF_TRUSTED_ORIGINS: http://impress-preprod.beta.numerique.gouv.fr,https://impress-preprod.beta.numerique.gouv.fr
DJANGO_CONFIGURATION: Production
DJANGO_ALLOWED_HOSTS: "*"
@@ -171,4 +180,4 @@ ingressMedia:
serviceMedia:
host: s3.margaret-hamilton.indiehosters.net
port: 443
port: 443

View File

@@ -8,6 +8,15 @@ backend:
argocd.argoproj.io/hook: PostSync
argocd.argoproj.io/hook-delete-policy: HookSucceeded
envVars:
AI_API_KEY:
secretKeyRef:
name: backend
key: AI_API_KEY
AI_BASE_URL:
secretKeyRef:
name: backend
key: AI_BASE_URL
AI_MODEL: meta-llama/Meta-Llama-3.1-70B-Instruct
DJANGO_CSRF_TRUSTED_ORIGINS: https://docs.numerique.gouv.fr
DJANGO_CONFIGURATION: Production
DJANGO_ALLOWED_HOSTS: "*"
@@ -171,4 +180,4 @@ ingressMedia:
serviceMedia:
host: s3.hedy-lamarr.indiehosters.net
port: 443
port: 443

View File

@@ -8,6 +8,15 @@ backend:
argocd.argoproj.io/hook: PreSync
argocd.argoproj.io/hook-delete-policy: HookSucceeded
envVars:
AI_API_KEY:
secretKeyRef:
name: backend
key: AI_API_KEY
AI_BASE_URL:
secretKeyRef:
name: backend
key: AI_BASE_URL
AI_MODEL: meta-llama/Meta-Llama-3.1-70B-Instruct
DJANGO_CSRF_TRUSTED_ORIGINS: http://impress-staging.beta.numerique.gouv.fr,https://impress-staging.beta.numerique.gouv.fr
DJANGO_CONFIGURATION: Production
DJANGO_ALLOWED_HOSTS: "*"

View File

@@ -58,6 +58,7 @@ releases:
- env.d/{{ .Environment.Name }}/values.impress.yaml.gotmpl
secrets:
- env.d/{{ .Environment.Name }}/secrets.enc.yaml
- env.d/{{ .Environment.Name }}/custom-secrets.enc.yaml
environments:
dev:
@@ -70,6 +71,11 @@ environments:
- version: 0.0.1
secrets:
- env.d/{{ .Environment.Name }}/secrets.enc.yaml
docs-ia:
values:
- version: 0.0.1
secrets:
- env.d/{{ .Environment.Name }}/secrets.enc.yaml
preprod:
values:
- version: 0.0.1

View File

@@ -19,3 +19,5 @@ stringData:
{{- end }}
OIDC_RP_CLIENT_ID: {{ .Values.oidc.clientId }}
OIDC_RP_CLIENT_SECRET: {{ .Values.oidc.clientSecret }}
AI_API_KEY: {{ .Values.aiApiKey }}
AI_BASE_URL: {{ .Values.aiBaseUrl }}

View File

@@ -24,14 +24,14 @@
<mj-text align="center">
<h1>
{% blocktrans %}
{{username}} shared a document with you !
{{sender_name}} shared a document with you !
{% endblocktrans %}
</h1>
</mj-text>
<!-- Main Message -->
<mj-text>
{% blocktrans %}
{{username}} invited you as an {{role}} on the following document :
{{sender_name_email}} invited you with the role "{{role}}" on the following document :
{% endblocktrans %}
<a href="{{link}}">{{document.title}}</a>
</mj-text>
@@ -52,7 +52,7 @@
/>
<mj-text>
{% blocktrans %}
Docs, your new essential tool for organizing, sharing and collaborate on your documents as a team.
Docs, your new essential tool for organizing, sharing and collaborating on your documents as a team.
{% endblocktrans %}
</mj-text>
<!-- Signature -->

View File

@@ -32,8 +32,7 @@
/* Global styles */
h1 {
color: #161616;
font-size: 1.5rem;
line-height: 1em;
font-size: 1.4rem;
font-weight: 700;
}