Compare commits

...

432 Commits

Author SHA1 Message Date
Manuel Raynaud
eb9487d5d7 (back) fix backend code related to multipage dev
During the multipage dev, the code base has changed a lot and rebase
after rebase it has come difficult to manage fixup commits. This commits
fix modification made that can be fixup in previous commits. The
persmission AccessPermission has been renamed in
ResourceWithAccessPermission and should be used in the
DocumentAskForAccessViewSet. A migration with the same dependency
exists, the last one is fixed. And a test didn't have removed an
abilitites.
2025-07-01 16:29:08 +02:00
Nathan Panchout
da40a84a51 wip 2025-07-01 15:49:27 +02:00
Nathan Panchout
48cacf4c99 (frontend) update test descriptions for clarity and consistency
- update tests description
- Corrected minor typos in test descriptions to enhance readability.
- Ensured that all test cases clearly convey their purpose and expected
outcomes.
2025-07-01 15:46:49 +02:00
Nathan Panchout
cfa11c5b35 (frontend) enhance document sharing and access management
- Introduced new utility functions for managing document sharing,
including `searchUserToInviteToDoc`, `addMemberToDoc`, and
`updateShareLink`.
- Updated existing tests to verify inherited share access and link
visibility features.
- Refactored document access handling in tests to improve clarity and
maintainability.
- Added comprehensive tests for inherited share functionalities,
ensuring proper role and access management for subpages.
2025-07-01 15:46:49 +02:00
Nathan Panchout
2e26ad2b09 (frontend) enhance document sharing and visibility features
- Added a new component `DocInheritedShareContent` to display inherited
access information for documents.
- Updated `DocShareModal` to include inherited share content when
applicable.
- Refactored `DocRoleDropdown` to improve role selection messaging based
on inherited roles.
- Enhanced `DocVisibility` to manage link reach and role updates more
effectively, including handling desynchronization scenarios.
- Improved `DocShareMemberItem` to accommodate inherited access logic
and ensure proper role management.
2025-07-01 15:46:48 +02:00
Nathan Panchout
dd742d9e6d (frontend) refactor document access API and remove infinite query
- Simplified the `getDocAccesses` function by removing pagination
parameters.
- Updated the `useDocAccesses` hook to reflect changes in the API
response type.
- Removed the `useDocAccessesInfinite` function to streamline document
access management.
2025-07-01 15:26:52 +02:00
Nathan Panchout
a0c56816f2 (frontend) enhance document management types and utilities
- Updated the `Access` and `Doc` interfaces to include new properties
for role management and document link reach.
- Introduced utility functions to handle document link reach and role,
improving the logic for determining access levels.
- Refactored the `isOwnerOrAdmin` function to simplify role checks for
document ownership and admin status.
2025-07-01 15:24:34 +02:00
Nathan Panchout
93def378a9 (backend) update Docker Hub workflow and fix migration conflict
- Added 'feature/doc-dnd' branch to the Docker Hub workflow to support
new feature development.
- Created a new migration to add the 'has_deleted_children' field to the
document model, enhancing the management of document states.
2025-07-01 15:24:34 +02:00
Nathan Panchout
6224e98774 (frontend) add new SVG assets and skeleton loading styles
- Introduced new SVG files for "desynchro" and "undo" icons to enhance
the user interface.
- Added a skeleton loading style in globals.css to improve the visual
experience during content loading.
2025-07-01 15:24:34 +02:00
Anthony LC
c438d4f081 ✏️(frontend) child document with different wording
We want to have a different wording when the child
document has no title, so we can distinguish
between the two cases.
2025-07-01 15:24:32 +02:00
Anthony LC
6188427b58 🔥(frontend) silent next.js error
The error modal since next.js 15 are quite intrusive.
We decided to hide them.
2025-07-01 15:23:43 +02:00
Nathan Panchout
1d087726fc (frontend) added new features for document management
- Created new files for managing subdocuments and detaching documents.
- Refactored API request configuration to use an improved configuration
type.
- Removed unnecessary logs from the ModalConfirmDownloadUnsafe
component.
2025-07-01 15:23:41 +02:00
Nathan Panchout
b852d19f43 (frontend) added subpage management and document tree features
New components were created to manage subpages in the document tree,
including the ability to add, reorder, and view subpages. Tests were
added to verify the functionality of these features. Additionally, API
changes were made to manage the creation and retrieval of document
children.
2025-07-01 15:23:25 +02:00
Nathan Panchout
a7a00a3087 (frontend) Added drag-and-drop functionality for document management
Added a new feature for moving documents within the user interface via
drag-and-drop. This includes the creation of Draggable and Droppable
components, as well as tests to verify document creation and movement
behavior. Changes have also been made to document types to include user
roles and child management capabilities.
2025-07-01 15:18:09 +02:00
Nathan Panchout
8ae112b43b (frontend) updated dependencies and added new packages
Added several new dependencies to the `package.json` file, including
`@dnd-kit/core`, `@dnd-kit/modifiers`, `@fontsource/material-icons`, and
`@gouvfr-lasuite/ui-kit`.
2025-07-01 15:18:07 +02:00
Nathan Panchout
e72317fbaa 🐛(back) keep info if document has deleted children
With the soft delete feature, relying on the is_leaf method from the
treebeard is not accurate anymore. To determine if a node is a leaf, it
checks if the number of numchild is equal to 0. But a node can have soft
deleted children, then numchild is equal to 0, but it is not a leaf
because if we want to add a child we have to look for the last child to
compute a correct path. Otherwise we will have an error saying that the
path already exists.
2025-07-01 15:17:32 +02:00
Samuel Paccoud - DINUM
507c9a583a (backend) add ancestors links definitions to document abilities
The frontend needs to display inherited link accesses when it displays
possible selection options. We need to return this information to the
client.
2025-07-01 15:17:32 +02:00
Samuel Paccoud - DINUM
d7ff740213 🐛(backend) fix link definition select options linked to ancestors
We were returning too many select options for the restricted link reach:
- when the "restricted" reach is an option (key present in the returned
  dictionary), the possible values for link roles are now always None to
  make it clearer that they don't matter and no select box should be
  shown for roles.
- Never propose "restricted" as option for link reach when the ancestors
  already offer a public access. Indeed, restricted/editor was shown when
  the ancestors had public/read access. The logic was to propose editor
  role on more restricted reaches... but this does not make sense for
  restricted since the role does is not taken into account for this reach.
  Roles are set by each access line assign to users/teams.
2025-07-01 15:17:32 +02:00
Samuel Paccoud - DINUM
4a44b44aec ♻️(backend) simplify roles by returning only the max role
We were returning the list of roles a user has on a document (direct
and inherited). Now that we introduced priority on roles, we are able
to determine what is the max role and return only this one.

This commit also changes the role that is returned for the restricted
reach: we now return None because the role is not relevant in this
case.
2025-07-01 15:17:32 +02:00
Samuel Paccoud - DINUM
94c372ea6e (backend) add ancestors links definitions to document abilities
The frontend needs to display inherited link accesses when it displays
possible selection options. We need to return this information to the
client.
2025-07-01 15:17:32 +02:00
Samuel Paccoud - DINUM
45ab75d638 🐛(backend) fix link definition select options linked to ancestors
We were returning too many select options for the restricted link reach:
- when the "restricted" reach is an option (key present in the returned
  dictionary), the possible values for link roles are now always None to
  make it clearer that they don't matter and no select box should be
  shown for roles.
- Never propose "restricted" as option for link reach when the ancestors
  already offer a public access. Indeed, restricted/editor was shown when
  the ancestors had public/read access. The logic was to propose editor
  role on more restricted reaches... but this does not make sense for
  restricted since the role does is not taken into account for this reach.
  Roles are set by each access line assign to users/teams.
2025-07-01 15:17:32 +02:00
Samuel Paccoud - DINUM
c807aded5c (backend) add max_role field to the document access API endpoint
The frontend needs to know what to display on an access. The maximum
role between the access role and the role equivalent to all accesses
on the document's ancestors should be computed on the backend.
2025-07-01 15:17:32 +02:00
Samuel Paccoud - DINUM
ad7a5235e1 ♻️(backend) simplify further select options on link reach/role
We reduce the number of options even more by treating link reach
and link role independently: link reach must be higher than its
ancestors' equivalent link reach and link role must be higher than
its ancestors' link role.

This reduces the number of possibilities but we decided to start
with the most restrictive and simple offer and extend it if we
realize it faces too many criticism instead of risking to offer
too many options that are too complex and must be reduced afterwards.
2025-07-01 15:17:32 +02:00
Samuel Paccoud - DINUM
62ff585539 🐛(backend) fix creating/updating document accesses for teams
This use case was forgotten when the support for team accesses
was added. We add tests to stabilize the feature and its security.
2025-07-01 15:17:32 +02:00
Samuel Paccoud - DINUM
8c6bf045cc (backend) add document path and depth to accesses endpoint
The frontend requires this information about the ancestor document
to which each access is related. We make sure it does not generate
more db queries and does not fetch useless and heavy fields from
the document like "excerpt".
2025-07-01 15:17:32 +02:00
Samuel Paccoud - DINUM
8e7651d5cd 🐛(backend) allow creating accesses when privileged by heritage
We took the opportunity of this bug to refactor serializers and
permissions as advised one day by @qbey: no permission checks in
serializers.
2025-07-01 15:17:30 +02:00
Samuel Paccoud - DINUM
12a229e0ab (backend) fix randomly failing test due to delay before check
There is a delay between the time the signature is issued and the
time it is checked. Although this delay is minimal, if the signature
is issued at the end of a second, both timestamps can differ of 1s.

> assert response["X-Amz-Date"] == timezone.now().strftime("%Y%m%dT%H%M%SZ")
AssertionError: assert equals failed '20250504T175307Z'  '20250504T175308Z'
2025-07-01 15:16:09 +02:00
Samuel Paccoud - DINUM
a3e3ede1f8 ♻️(backend) stop requiring owner for non-root documents
If root documents are guaranteed to have a owner, non-root documents
will automatically have them as owner by inheritance. We should not
require non-root documents to have their own direct owner because
this will make it difficult to manage access rights when we move
documents around or when we want to remove access rights for someone
on a document subtree... There should be as few overrides as possible.
2025-07-01 15:16:09 +02:00
Samuel Paccoud - DINUM
46ac2e11ed (backend) add max ancestors role field to document access endpoint
This field is set only on the list view when all accesses for a given
document and all its ancestors are listed. It gives the highest role
among all accesses related to each document.
2025-07-01 15:16:09 +02:00
Samuel Paccoud - DINUM
e0d6271677 fixup! ♻️(backend) optimize refactoring access abilities and fix inheritance 2025-07-01 15:16:09 +02:00
Samuel Paccoud - DINUM
9e9efa197a ♻️(backend) optimize refactoring access abilities and fix inheritance
The latest refactoring in a445278 kept some factorizations that are
not legit anymore after the refactoring.

It is also cleaner to not make serializer choice in the list view if
the reason for this choice is related to something else b/c other
views would then use the wrong serializer and that would be a
security leak.

This commit also fixes a bug in the access rights inheritance: if a
user is allowed to see accesses on a document, he should see all
acesses related to ancestors, even the ancestors that he can not
read. This is because the access that was granted on all ancestors
also apply on the current document... so it must be displayed.

Lastly, we optimize database queries because the number of accesses
we fetch is going up with multi-pages and we were generating a lot
of useless queries.
2025-07-01 15:16:07 +02:00
Samuel Paccoud - DINUM
9b29a0b693 (backend) add computed link reach and role to document API
On a document, we need to display the status of the link (reach and
role) taking into account the ancestors link reach/role as well as
the current document.
2025-07-01 15:15:17 +02:00
Samuel Paccoud - DINUM
b85f1d1737 (backend) add ancestors link reach and role to document API
On a document, we need to display the status of the link (reach and
role) as inherited from its ancestors.
2025-07-01 15:15:17 +02:00
Samuel Paccoud - DINUM
4474fc8ddb ♻️(backend) simplify roles by returning only the max role
We were returning the list of roles a user has on a document (direct
and inherited). Now that we introduced priority on roles, we are able
to determine what is the max role and return only this one.

This commit also changes the role that is returned for the restricted
reach: we now return None because the role is not relevant in this
case.
2025-07-01 15:15:16 +02:00
Samuel Paccoud - DINUM
2f980e880a (backend) fix randomly failing test on user search
The user account created to query the API had a random email
that could randomly interfere with our search results.
2025-07-01 15:15:16 +02:00
Samuel Paccoud - DINUM
6edb92ee62 ♻️(backend) refactor get_select_options to take definitions dict
This will allow us to simplify the get_abilities method. It is also
more efficient because we have computed this definitions dict and
the the get_select_options method was doing the conversion again.
2025-07-01 15:15:14 +02:00
Samuel Paccoud - DINUM
ee8f61a858 (backend) give an order to choices
We are going to need to compare choices to materialize the fact that
choices are ordered. For example an admin role is higer than an
editor role but lower than an owner role.

We will need this to compute the reach and role resulting from all
the document accesses (resp. link accesses) assigned on a document's
ancestors.
2025-07-01 15:13:40 +02:00
Samuel Paccoud - DINUM
40785f381a (backend) we want to display ancestors accesses on a document share
The document accesses a user have on a document's ancestors also apply
to this document. The frontend needs to list them as "inherited" so we
need to add them to the list.
Adding a "document_id" field on the output will allow the frontend to
differentiate between inherited and direct accesses on a document.
2025-07-01 15:13:40 +02:00
Samuel Paccoud - DINUM
149f711f60 ♻️(backend) factorize document query set annotation
The methods to annotate a document queryset were factorized on the
viewset but the correct place is the custom queryset itself now that
we have one.
2025-07-01 15:13:40 +02:00
Samuel Paccoud - DINUM
b5dad9eb1a ♻️(backend) refactor resource access viewset
The document viewset was overriding the get_queryset method from its
own mixin. This was a sign that the mixin was not optimal anymore.
In the next commit I will need to complexify it further so it's time
to refactor the mixin.
2025-07-01 15:13:40 +02:00
Samuel Paccoud - DINUM
8c714bfdc6 ♻️(backend) remove different reach for authenticated and anonymous
If anonymous users have reader access on a parent, we were considering
that an edge use case was interesting: allowing an authenticated user
to still be editor on the child.

Although this use case could be interesting, we consider, as a first
approach, that the value it carries is not big enough to justify the
complexity for the user to understand this complex access right heritage.
2025-07-01 15:13:40 +02:00
Samuel Paccoud - DINUM
55ea7f8375 (backend) add ancestors links definitions to document abilities
The frontend needs to display inherited link accesses when it displays
possible selection options. We need to return this information to the
client.
2025-07-01 15:13:36 +02:00
Samuel Paccoud - DINUM
abddb4d585 🐛(backend) fix link definition select options linked to ancestors
We were returning too many select options for the restricted link reach:
- when the "restricted" reach is an option (key present in the returned
  dictionary), the possible values for link roles are now always None to
  make it clearer that they don't matter and no select box should be
  shown for roles.
- Never propose "restricted" as option for link reach when the ancestors
  already offer a public access. Indeed, restricted/editor was shown when
  the ancestors had public/read access. The logic was to propose editor
  role on more restricted reaches... but this does not make sense for
  restricted since the role does is not taken into account for this reach.
  Roles are set by each access line assign to users/teams.
2025-07-01 15:11:06 +02:00
soyouzpanda
6c3850b22b (frontend) support _FILE environment variables for secrets
Allow configuration variables that handles secrets to be read from a
file given in an environment variable.
2025-07-01 10:47:55 +02:00
soyouzpanda
31e8ed3a00 (backend) support _FILE environment variables for secrets
Allow configuration variables that handles secrets, like
`DJANGO_SECRET_KEY` to be able to read from a file which is given
through an environment file.

For example, if `DJANGO_SECRET_KEY_FILE` is set to
`/var/lib/docs/django-secret-key`, the value of `DJANGO_SECRET_KEY` will
be the content of `/var/lib/docs/django-secret-key`.
2025-07-01 10:32:55 +02:00
Manuel Raynaud
7e63e9e460 ♻️(back) exclude /admin from CSP rules
We have to exclude the /admin prefix to allow loading static files when
the django admin is used.
2025-06-30 14:46:01 +02:00
Anthony LC
388f71d9d0 (frontend) button access request on share modal
When a document is in public or connected mode,
users can now request access to the document.
2025-06-30 12:13:28 +02:00
Anthony LC
2360a832af (frontend) add access request on doc share modal
Add the access request to the document
share modal, allowing admin to see and manage
access requests directly from the modal interface.
2025-06-30 12:13:28 +02:00
Anthony LC
411d52c73b ♻️(frontend) improve separation of concerns in DocShareModal
Improve separation of concerns in the DocShareModal
component.
The member and invitation list are now
in a separate component.
It will help us to integrate cleanly the
request access list.
2025-06-30 12:13:28 +02:00
Anthony LC
394f91387d (backend) send email to admins when user ask for access
When a user requests access to a document, an
email is sent to the admins and owners of the
document.
2025-06-30 12:13:27 +02:00
Anthony LC
878de08b1e (frontend) integrate doc access request
When a user is redirected on the 403 page,
they can now request access to the document.
2025-06-30 12:13:27 +02:00
Manuel Raynaud
d33286019c (back) accept for a owner the request to access a document
Add the action accepting a request to access a document. It is possible
to override the role from the request and also update an existing
DocumentAccess
2025-06-30 12:13:26 +02:00
Manuel Raynaud
c2e46fa9e2 (back) document as for access CRUD
We introduce a new model for user wanted to access a document or upgrade
their role if they already have access.
The viewsets does not implement PUT and PATCH, we don't need it for now.
2025-06-30 12:13:26 +02:00
Manuel Raynaud
2e1b112133 🚨(back) remove unused ruff ignore rule
A ruff ignore rule was present in the factories module. But this rule is
not used in the file so we can safely remove it.
2025-06-30 10:43:58 +02:00
renovate[bot]
8f7ac12ea1 ⬆️(dependencies) update python dependencies 2025-06-30 10:43:58 +02:00
Manuel Raynaud
dfdfe83db5 (back) install and configure django csp (#1085)
We want to protect all requests from django with content security
policy header. We use the djang-csp library and configure it with
default values.

Fixes #1000
2025-06-30 08:42:48 +00:00
Anthony LC
4ae757ce93 🔥(frontend) remove Beta from logo
Docs got homologated, so we can remove the beta
logo from the DSFR theme.
2025-06-27 18:22:59 +02:00
Manuel Raynaud
6964686f7c 🔧(back) remove usage of deprecated db engine
The db engine postgresql_psycopg2 does not exists anymore in django but
for BC compat it is possible to use it in the configuration and it is
replace by postgresql at runtime. We changed this settings to use the
good one.
2025-06-27 16:03:09 +00:00
Manuel Raynaud
45bbffdf9f (back) allow to disable checking unsafe mimetype on attachment upload
We added the possibility to scan all uploaded files with an anti malware
solution. Depending the backend used, we want to give the possibility to
check the file mimtype to determine if this one is tagged as unsafe or
not. To this you can set the environment variable
DOCUMENT_ATTACHMENT_CHECK_UNSAFE_MIME_TYPES_ENABLED to False. The
default value is True.
2025-06-27 15:31:15 +00:00
Anthony LC
95a55e7805 (e2e) reduce flakiness in e2e tests
Flakiness in e2e tests has been reduced by:
- Adding waits for media-check processing in image tests.
- Ensuring that slash menu resets are handled
correctly to avoid flakiness.
- Wait for the Download button to be stable before clicking
2025-06-27 16:50:10 +02:00
Sylvain Zimmer
e9ac36e811 📝(readme) fix some small issues in the README
Fix some typo and small issues in the README file.
2025-06-27 15:10:14 +02:00
Stephan Meijer
d8294ee11d fix: Makefile failing on run-frontend-development (#1104)
See #1103

Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
2025-06-27 12:35:55 +00:00
Anthony LC
00009ecc16 🔧(conf) add server to server api tokens to common
We have the e2e test "it creates a doc server way"
that is quite complicated to run locally, because
it requires the `DJANGO_SERVER_TO_SERVER_API_TOKENS`
environment variable to be set in "env.d/development/common".
We moved `DJANGO_SERVER_TO_SERVER_API_TOKENS` from
"env.d/development/common.e2e.dist" to
"env.d/development/common.dist", by doing so,
this variable will be set by default in the
"env.d/development/common" file, the test will now run
without any additional configuration.
2025-06-26 17:09:08 +02:00
Anthony LC
9b0676ec15 (jest) fix window.location mock
We upgraded to jest 30.0.3.
This upgrade updated jsdom and jsdom now do not
allows to mock window.location.
See: https://github.com/jsdom/jsdom/issues/3492
This commit fixes this issue.
2025-06-26 16:49:13 +02:00
Anthony LC
9f222bbaa3 ⬇️(dependencies) downgrade to docx 9.5.0
Prob compatibility issue with docx 9.5.1 and
BlockNote. We downgrade to 9.5.0 for now until
BlockNote is updated to support docx 9.5.1.
2025-06-26 16:47:51 +02:00
renovate[bot]
f0b253f0ff ⬆️(dependencies) update js dependencies 2025-06-26 12:14:34 +00:00
Timothee Gosselin
1e76e6e04c Documentation for self-hosting with docker compose (#855)
## Purpose

Make self hosting of Docs easier with an example of a deployment
procedure with docker compose and document how to configure Docs.

While https://github.com/suitenumerique/docs/pull/583 propose an easy
way to deploy Docs with docker and Make, here we describe more in
details the various steps and requirements to deploy Docs.

## Proposal

- [x] example to deploy and configure keycloak
- [x] example to deploy and configure minio
- [x] example to configure proxy and certs
- [x] example to deploy and configure Docs

## Improvements
- [x] Rephrase description of environment variables and categorize
- [x] Use template for nginx conf  

Fixes https://github.com/suitenumerique/docs/issues/561
Supersedes https://github.com/suitenumerique/docs/pull/583

 A one liner quick start could be a nice addition:
- [ ] merge all services in a single compose
- [ ] scripts to generate secrets

Signed-off-by: unteem <timothee@indie.host>
2025-06-25 13:02:08 +00:00
Anthony LC
a71453206b 🐛(env) update yprovider env for local development
In local development the notification to
the yprovider server was not working anymore
because of a recent change in the container name.
We adapt the env variables to match the new
container name.
2025-06-24 16:08:23 +02:00
lebaudantoine
71cd016d4d ️(frontend) optimize document fetch error handling
Reduce unnecessary fetch requests when retrieving documents with permission
or authentication issues. Previous implementation was triggering multiple
document requests despite having sufficient error information from initial
attempt to determine appropriate user redirection.

Additionally, fix issue where resetting the auth cache was triggering redundant
authentication verification requests. The responsibility for checking auth
status should belong to the 401 page component on mount, rather than being
triggered by cache resets during error handling.

Known limitations:
- Not waiting for async  function completion makes code harder to
 maintain
- Added loading spinner as temporary solution to prevent UI flicker
- Future improvement should implement consistent error-based redirects rather
 than rendering error messages directly on document page
2025-06-24 15:50:02 +02:00
lebaudantoine
2a7ffff96d ️(frontend) prevent authentication retry on 401 responses
Stop retry attempts when receiving 401 Unauthorized from /me endpoint since
this clearly indicates authentication status. The original purpose of the /me
call is simply to determine if user is authenticated, and a 401 provides
sufficient information.

Prevents unnecessary network requests caused by React Query's automatic retry
behavior when re-raising exceptions, which was hiding the 401 status. Improves
performance and reduces server load during authentication failures.
2025-06-24 15:46:48 +02:00
Erik Duxstad
ff8275fb4e 📝(self-hosted) update collaboration vars (#1075)
Remove the `auth-url` annotation and add the
`COLLABORATION_BACKEND_BASE_URL` variable, introduced in 3.0.0.

Mount the development CA to the yProvider container to allow
TLS connections with the backend.

Fix the mount path for development CA in the backend container.

Signed-off-by: eduxstad <eduxstad@gmail.com>
2025-06-24 06:21:29 +00:00
Bastien
c3f81c2b62 📝(docs) minor fixes to docs/env.md (#1086)
Replaces https://github.com/suitenumerique/docs/pull/941

Signed-off-by: Bastien Guerry <bastien.guerry@code.gouv.fr>
2025-06-20 15:53:39 +02:00
Manuel Raynaud
c7261cf507 🔧(front) configure x-frame-options to DENY in nginx conf (#1084)
The API has the response header x-frame-options configure to DENY and
nothing is configure in the nginx configuring managing the frontend
application. We want to have the same value. The header is added on all
locations.
2025-06-19 15:36:57 +02:00
Anthony LC
e504f43611 👥(github) update pull request template
We added a new section to the pull request
template to ensure that contributors
follow the correct process for submitting
pull requests.
2025-06-17 14:06:55 +02:00
Anthony LC
3ad6d0ea12 📝(project) add system-requirement doc
Add a new document detailing the system
requirements for the project.
2025-06-17 14:06:55 +02:00
Anthony LC
9e8a7b3502 📝(project) add troubleshoot doc
Add a troubleshooting document to help users
resolve common issues.
2025-06-17 14:06:54 +02:00
Manuel Raynaud
05db9c8e51 🤡(demo) change dev users email to remove invalid domain extension
The domain extension (.e2e) used in the demo for users are not validated
anymore by the django EmailValidator. We have to change it to a valid
one.
2025-06-17 13:35:41 +02:00
Manuel Raynaud
7ed33019c2 ⬆️(back) upgrade django to version 5.2
Django 5.2 is now mature enough and we can use it in production.
In some tests the number of sql queries is increasing. This is because
the `full_clean` method called in the `save` method on all our models is
creating a transaction, so a savepoint and release is added.
We also fix deprecated warning in this commit.
2025-06-17 12:20:19 +02:00
Samuel Paccoud
a99c813421 📌(yjs) stop pinning node to minor version on yjs docker image (#1005)
We want to build the yjs Docker image with the latest minor version in
order to avoid outdated images.
2025-06-17 09:43:05 +00:00
Jacques ROUSSEL
a83902a0d4 🚸(helm) improve helm chart
Our Helm chart wasn't suitable for use with Helm alone because jobs
remained after deployment. We chose to configure ttlSecondsAfterFinished
to clean up jobs after a period of time.
2025-06-16 16:05:48 +02:00
renovate[bot]
080f855083 ⬆️(dependencies) update python dependencies 2025-06-16 03:24:19 +00:00
Anthony LC
90d94f6b7a ⬆️(frontend) Bump brace-expansion
Bumps [brace-expansion](https://github.com/juliangruber/brace-expansion)
from 2.0.1 to 2.0.2.
Bumps [brace-expansion](https://github.com/juliangruber/brace-expansion)
from 1.1.11 to 1.1.12.

Signed-off-by: dependabot[bot] <support@github.com>
2025-06-13 15:30:04 +02:00
Simon Ser
f97ab51c8e 🧑‍💻(docker) add y-provider config to development environment
Without this, YdocConverter throws an error when developping.
2025-06-13 10:53:22 +02:00
Manuel Raynaud
ba4f90a607 🧑‍💻(compose) remove --no-cache to build images by default
In order to speed the rebuild of images, the --no-cache option is
removed by default. If we want to build the images without cache, the
cache paramter must be used.
2025-06-13 10:53:09 +02:00
Manuel Raynaud
6c16e081de 🧑‍💻(docker) create a e2e compose configuration
We want to run the e2e tests using the frontend and y-provider
production images. We created a dedicated compose file adding just
missing services. These services are built in the CI.
2025-06-13 10:38:53 +02:00
Manuel Raynaud
56a945983e ♻️(docker) rename docker-compose.yaml in compose.yml
The usage of docker-compose.yaml file is deprecated, we can rename it in
compose.yml
2025-06-13 10:38:53 +02:00
Manuel Raynaud
4fbbead405 🧑‍💻(compose) build and run y-provider in dev mode
To have a better developer experience, the y-provider service run in dev
mode, allowing hot reload when a file is modified. To avoid issue with
shared node_modules, they are mounted in a separated volume to not have
then in the local directory.
2025-06-13 10:38:53 +02:00
Manuel Raynaud
9a212400a0 🔥(compose) remove app and celery services
`app` and `celery` services are not used when we run the compose
configuration. The compose file is only used for development purpose.
2025-06-13 10:38:52 +02:00
Manuel Raynaud
f07fcd4c0d 🔧(docker) add a service in compose to frontend development
We want a serice in compose starting the frontend application in
development mode. We want to take the advantage of the hot reload
module, so the sources are mounted inside the container.
2025-06-13 10:38:52 +02:00
Appryll
4fc49d5cb2 ️(frontend) Set page titles for 403 and 404 errors
Set the page titles for the 403 and 404 error
pages to improve user experience and accessibility.
2025-06-11 16:36:53 +02:00
Anthony LC
0fd16b4371 💄(frontend) add spacing bottom on editor
We add spacing bottom on editor to
avoid the last editing line being to close to
the bottom.
2025-06-11 13:08:33 +02:00
Anthony LC
fbb2799050 🔧(git) set LF line endings for all text files
Windows users are by default using CRLF line endings,
which can cause issues with some tools and
environments. This commit sets the `.gitattributes`
file to enforce LF line endings for all text
files in the repository.
2025-06-11 13:08:33 +02:00
Manuel Raynaud
afbb4b29dc 🩹(backend) default CORS_ALLOW_ALL_ORIGINS to False
The settings CORS_ALLOW_ALL_ORIGINS was set to True by default.

This error is inherited from a old mistake made back in the days while
working on the initial impress demo.

This is not something we want, this should be only allowed in
development. We change the value in all the manifests in order to have
the desired behavior in non development environments.
2025-06-11 09:55:28 +00:00
Anthony LC
db63ebd0c8 🐛(frontend) fix meta title
The meta title was flickering, it was adding the
doc title, then it was coming back to the default
title.
This was due to the way the next Head component
render data.
We now use a more stable way to set the title.
2025-06-11 10:21:53 +02:00
Anthony LC
c5f018e03e 💄(frontend) adapt some style
- editor block padding only when background
- increase icon shadow grid
2025-06-11 10:03:32 +02:00
Anthony LC
1c93fbc007 🐛(frontend) fix multiple EmojiPicker
emoji-mart is used to display emojis in the editor.
It is used by the callout block and by
Blocknotes editor. The problem is that the emoji-mart
is a singleton, so if Blocknotes components init
the emoji-mart first, the picker in the callout block
will not display correctly.
This commit fixes the issue by initializing
the emoji-mart in the callout block first.
2025-06-11 10:03:32 +02:00
Anthony LC
d811e3c2fc 🐛(frontend) table of content disappearing
The table of content was disappearing when the user
was looking the version history then came back to
the main document.
This commit fixes this issue.
2025-06-11 09:45:42 +02:00
Anthony LC
fe5fda5d73 ✏️(project) fix typo
Fix and improve typos in the codebase.
2025-06-11 09:10:22 +02:00
Simon Ser
bf66265125 🙈(docker) add .next to .dockerignore
We don't want to copy this over to the Docker daemon, since this
directory can be quite large.
2025-06-11 08:29:28 +02:00
renovate[bot]
ce329142dc ⬆️(dependencies) update python dependencies 2025-06-10 07:58:32 +00:00
renovate[bot]
f8cff43dac ⬆️(dependencies) update requests to v2.32.4 [SECURITY] 2025-06-10 07:35:04 +00:00
renovate[bot]
f5b2c27bd8 ⬆️(dependencies) update django to v5.1.10 [SECURITY] 2025-06-06 17:21:22 +00:00
Anthony LC
62433ef7f1 ♻️(i18n) adapt script to major upgrade of yargs
"yargs" dependency has been updated to version 18.0.0,
which causes breaking changes in the script.
2025-06-05 10:58:59 +02:00
Anthony LC
bc0824d110 🚨(frontend) fix linter warning react-query
React-query change the types of some methods, which causes
linter warnings. This commit updates the affected methods
to match the new types.
2025-06-05 10:46:53 +02:00
Anthony LC
fa653c6776 🏷️(CI) add automated label to renovate
Renovate provides automated pull requests, so let's
use a label to identify them easily.
2025-06-05 10:46:53 +02:00
Anthony LC
d12f942d29 ⬆️(project) bump project to node 22
"yargs" dependency requires node 22, so we
update the project to use it.
Node 22 is the latest LTS version, so this is a
good time to do it.
2025-06-05 10:46:53 +02:00
renovate[bot]
62f85e7d24 ⬆️(dependencies) update js dependencies 2025-06-05 10:46:53 +02:00
Manuel Raynaud
65cc088a17 ⬆️(compose) upgrade node image to version 22
We node service in doecker compose can be a helper to use node locally
without installing it. Docs requires at least node 22 so we upgrade it
to node 22.
2025-06-05 10:28:47 +02:00
rvveber
94e99784f3 (tests) Add & adapt language tests
- Language will only be changed if different from current language
- Added test for custom translations

Signed-off-by: Robin Weber <weber@b1-systems.de>
2025-06-03 17:35:52 +02:00
rvveber
fa83955a77 ♻️(frontend) Refactor language-related code
- Refactors "useTranslationsCustomizer" to "useCustomTranslations"
- Refactors "useLanguageSynchronizer" to "useSynchronizedLanguage"
- Refactors "LanguagePicker" to better reflect its component role
- Refactors "LanguagePicker" to use "useSynchronizedLangue"
- Removes unused "useChangeUserLanguage"
- To change the user language, use "useAuthMutation" instead

Signed-off-by: Robin Weber <weber@b1-systems.de>
2025-06-03 17:35:52 +02:00
rvveber
5962f7aae1 ♻️(frontend) Separate mutations from queries for auth logic
Introduces dedicated mutations
(for authentication/user operations)
separating them from queries to align with best practices
for data fetching and state management.

Queries remain responsible for READ operations, while mutations
now handle CREATE, UPDATE, and DELETE actions (for user data)
improving separation of concerns.

Signed-off-by: Robin Weber <weber@b1-systems.de>
2025-06-03 17:35:52 +02:00
rvveber
dc06315566 📝(documentation) adds customization for translations
Part of customization PoC

Signed-off-by: Robin Weber <weber@b1-systems.de>
2025-06-03 17:35:52 +02:00
rvveber
f4ad26a8fa (frontend) Adds customization for translations
Part of customization PoC

Signed-off-by: Robin Weber <weber@b1-systems.de>
2025-06-03 17:35:52 +02:00
renovate[bot]
d952815932 ⬆️(dependencies) update python dependencies 2025-06-02 05:09:03 +00:00
renovate[bot]
cde64ed80a ⬆️(dependencies) update js dependencies 2025-05-26 06:39:40 +00:00
renovate[bot]
cfd88d0469 ⬆️(dependencies) update python dependencies 2025-05-26 01:55:36 +00:00
virgile-dev
5e45fec296 📝(doc) fix path to env doc on readme (#1007)
The path lead to a 404

Signed-off-by: virgile-dev <virgile.deville@beta.gouv.fr>
2025-05-25 17:01:29 +00:00
Anthony LC
393e7a06e2 🔖(minor) release 3.3.0
Added:
- (backend) add endpoint checking media status
- (backend) allow setting session cookie age via env var
- (backend) allow theme customnization using a
configuration file
- (frontend) Add a custom callout block to the editor
- 🚩(frontend) version MIT only
- (backend) integrate maleware_detection from django-lasuite
- 🏗️(frontend) Footer configurable
- 🩺(CI) add lint spell mistakes
- (frontend) create generic theme
- 🛂(frontend) block edition to not connected users
- 🚸(frontend) Let loader during upload analyze
- 🚩(frontend) feature flag on blocking edition

Changed:
- 📝(frontend) Update documentation
- (frontend) Improve tests coverage
- ⬆️(docker) upgrade backend image to python 3.13
- ⬆️(docker) upgrade node images to alpine 3.21

Fixed:
- 🐛(y-provider) increase JSON size limits for
transcription conversion

Removed:
- 🔥(back) remove footer endpoint
2025-05-23 11:41:24 +02:00
AntoLC
f1af87baf8 🌐(i18n) update translated strings
Update translated files with new translations
2025-05-23 11:41:24 +02:00
Anthony LC
f851ef2d85 ⬆️(dependencies) bump blocknote to 0.30.1
A bunch of fixes are in this pacth release.
Better to update now before we release to 3.3.0.
2025-05-23 11:08:07 +02:00
Anthony LC
252ab6a586 ✏️(frontend) change antivirus fail sentence
The message was not accurate when the analizer failed.
We improved the message to be more accurate.
2025-05-23 11:08:07 +02:00
Anthony LC
cf2a02c8de 🚩(frontend) feature flag on blocking edition
If users were not connected to the collaboration
server, they were not be able to edit documents.

We decided to add a feature flag on this feature
as it can be quite restrictive.
We can now enable or disable this feature at runtime
thanks to the env variable
"COLLABORATION_WS_NOT_CONNECTED_READY_ONLY".
2025-05-23 11:08:07 +02:00
Anthony LC
d87a2ed4eb 🔥(helm) remove useless footer config
Remove:
- FRONTEND_FOOTER_FEATURE_ENABLED
- FRONTEND_FOOTER_URL
2025-05-22 15:27:39 +02:00
Anthony LC
c9d053d1c0 💄(frontend) add generic favicon
The favicons were still with the dsfr color.
We added the generic favicon in the assets folder.
The favicon can be a url loaded from the theme,
so when Drive will be running, we will be able
to store the dsfr favicons there, and remove them
from the repo.
2025-05-22 15:27:38 +02:00
Anthony LC
b5f0f06ea3 💄(frontend) desaturate images system for generic theme
We want to desaturate the images system in the
generic theme to make them less colorful and more
in line with the overall theme.
We added a special class to the images
that need to be desaturated. Other property
then desaturated can be apply depending on the theme.
2025-05-22 15:27:38 +02:00
Anthony LC
36b0ff9f63 (frontend) create generic theme
By default Docs will not be on the dsfr theme but
on the generic theme. La Gaufre is part of the dsfr
theme and is removed from the generic theme.
Same for the "beta" keyword and the "proconnect"
buttons.
2025-05-22 15:27:38 +02:00
Anthony LC
7a383957a7 🔥(frontend) remove legal pages
Legal pages are not needed anymore in the application.
In the dsfr instances, the legal pages will be
displayed on a Docs pages.
We let the users of Docs managing the legal pages
on their own instances.
2025-05-22 14:07:41 +02:00
Anthony LC
b5630359ee 🏗️(frontend) Footer configurable
To have different footer per instance the
content of the footer is now configurable
from the theme customization file.

See THEME_CUSTOMIZATION_FILE_PATH env var.
2025-05-22 14:07:41 +02:00
Anthony LC
310154815b ♻️(e2e) improve config testcases
Improve config testcases:
- let THEME_CUSTOMIZATION_FILE_PATH to be set to
check the default value
- add helper function overrideConfig
2025-05-22 14:07:41 +02:00
Anthony LC
2733785016 🚨(linter) add ignore pattern on no-unused-vars rule
The rule @typescript-eslint/no-unused-vars didn't
have a ignore pattern. A ignore pattern can be
usefull in some cases.
2025-05-22 14:07:41 +02:00
Manuel Raynaud
99ba414d88 🔧(back) add docs.security to logging settings
In the malware_detection callback we are using a different logger named
docs.security. We want to configure a logger in the logging settings
handling it.
2025-05-22 13:53:27 +02:00
Manuel Raynaud
41631b5b70 ⬆️(backend) upgrade django-lasuite to version 0.0.9
We need version 0.0.9 to reduce the time to have a JCOP analysis result.
2025-05-22 13:53:27 +02:00
Anthony LC
6ca654bf1a 🚸(frontend) let loader until resource ready
The backend can analyze the upload file, this take
time, so we need to show a loader
until the backend finish the analysis.
2025-05-22 13:53:27 +02:00
Manuel Raynaud
074585337b ♻️(back) return the media-check url on the attachment_upload response
We want to have the media-check url returned on the attachment-upload
response instead of the media url directly. The front will know the
endpoint to use to check the media status.
2025-05-22 13:39:44 +02:00
Manuel Raynaud
f1b398e1ae (back) add endpoint checking media status
With the usage of a malware detection system, we need a way to know the
file status. The front will use it to display a loader while the analyse
is not ended.
2025-05-22 13:39:44 +02:00
Jacques ROUSSEL
d1f73f18cd 🔒️(front) improve docker image security
Cyberwatch reported security issues with the frontend Docker image.
2025-05-22 11:16:57 +02:00
lebaudantoine
3f2d84bf62 🐛(y-provider) increase JSON size limits for transcription conversion
Problem:
- Default Express JSON parser limit (100kb) is insufficient for larger
 transcription files
- 2-hour audio transcriptions slightly exceed the 100kb limit, causing request
 failures

Solution:
- Implemented custom middleware to apply different JSON parser configurations
 based on route
- Applied 500kb limit specifically for transcription conversion endpoints
- Maintained default limits for all other routes to preserve security

Technical notes:
- Could not find a built-in Express solution to specify parser config per route
- Custom middleware conditionally applies the appropriate parser configuration
2025-05-21 15:31:49 +02:00
lebaudantoine
7b9c362d38 🐛(tilt) update certificate path for Python 3.13 upgrade
Fix certificate directory reference that still pointed to Python 3.12 folder
after upgrading to Python 3.13. Resolves certificate verification errors in
tilt stack caused by incorrect certificate location.
2025-05-21 12:53:41 +02:00
virgile-dev
bf999979d2 📝(doc) update xl packages warning (#985)
So that people know how to use the PUBLISH_AS_MIT variable

Signed-off-by: virgile-dev <virgile.deville@beta.gouv.fr>
2025-05-20 21:10:54 +02:00
renovate[bot]
09d3ff3754 ⬆️(dependencies) update python dependencies 2025-05-19 12:21:04 +00:00
Samuel Paccoud - DINUM
6e5d005dee (backend) allow setting session cookie age via env var
We want to be able to increase the duration of the cookie session
by setting an environment variable.
2025-05-19 13:57:30 +02:00
Anthony LC
6377c8fcca ✈️(frontend) allow editing when offline
When the user is offline, we allow editing the
document in the editor.
Their is not a reliable way to know if the user is
offline or online except by doing a network request
and checking if an error is thrown or not.
To do so, we created the OfflinePlugin inherited
from the WorkboxPlugin.
It will inform us if the user is offline or online.
We then dispatch the information to our application
thanks to the useOffline hook.
2025-05-19 12:36:32 +02:00
Anthony LC
3c8cacc048 🛂(frontend) block edition to not connected users
If an editor is working on a shared document but
is not connected to the collaborative server
we are now blocking the edition.
It is to avoid none connected users to
overwrite the document with connected
users.
2025-05-19 12:36:31 +02:00
virgile-dev
598fb4fa27 📝(doc) update issue templates (#976)
Mention of Impress instead of Docs is confusing. Also added some
automatic labelling.

Signed-off-by: virgile-dev <virgile.deville@beta.gouv.fr>
2025-05-19 09:14:00 +00:00
Anthony LC
51618ad081 📌(dependencies) add hocuspocus to renovate ignored list
hocuspocus > 3.0.0 brings breaking changes.
Let's add it in the renovate.json ignored list
until we decide to upgrade it.
2025-05-19 09:19:37 +02:00
renovate[bot]
8109d5ba08 ⬆️(dependencies) update js dependencies 2025-05-19 09:17:33 +02:00
virgile-dev
e4d0179bbe 📝(doc) readme update (#974)
Added badges so people can assess the repo activity easily. 
Also update the paragraph about AGPL XL packages since this was merged :
https://github.com/suitenumerique/docs/pull/911

---------

Signed-off-by: virgile-dev <virgile.deville@beta.gouv.fr>
2025-05-16 17:48:57 +00:00
Manuel Raynaud
9d3dfb6de7 ⬆️(docker) upgrade node images to alpine 3.21
We need to upgrade our images to alpine 3.21 in order to fix a CVE
related to libxml2. We also upgrade node to version 24
2025-05-16 15:55:33 +02:00
Manuel Raynaud
0da042f887 ⬆️(docker) upgrade backend image to python 3.13
Python 3.13 is now stable, our libraries are compatible with it. We also
upgrade the alpine version used in order to fix CVE related to libxml2
2025-05-16 15:55:32 +02:00
Anthony LC
6cd0cd0689 ⬆️(dependencies) gouvfr-lasuite/ui-kit to 0.6.0
Upgrade @gouvfr-lasuite/ui-kit from 0.5.0 to 0.6.0.
Some properties have been removed in 0.5.0, which
causes the design of the app to be broken.
Version 0.6.0 has as well some breaking changes,
about the "logo" properties that are not available anymore.
We fix them in this commit.
2025-05-16 12:35:16 +02:00
Anthony LC
10b088599c 🐛(frontend) fix svg export
Last upgrade of Blocknote to 0.30.0 broke the SVG
export. The previewWidth can be undefined, which causes the
export to fail. This commit adds a fallback
width in case previewWidth is undefined.
2025-05-16 11:12:30 +02:00
Anthony LC
62d1bc6473 🐛(frontend) redirect to /home
The page '/login' was replaced with '/home',
but some users may still have the old URL in their
bookmarks, it can create a loop during the
authentication process.
We redirect the user to '/home' if they try to access
'/login' page, it will prevent edge cases.
2025-05-16 11:12:30 +02:00
Anthony LC
fc1d33268c ⬆️(dependencies) update js dependencies 2025-05-16 11:12:30 +02:00
virgile-dev
95833fa5ec 📝(documentation) add banner to readme (#970)
Implement suggestions made by @xibe in
[#848](https://github.com/suitenumerique/docs/pull/848) and
[#849](https://github.com/suitenumerique/docs/pull/849)

Signed-off-by: virgile-dev <virgile.deville@beta.gouv.fr>
2025-05-16 11:10:05 +02:00
Anthony LC
dd6e0b5072 📝(project) add missing env var to env.md
Update the documentation to include the
missing environment variables.
The missing environment variables are involved
in the build process of the frontend image.
2025-05-13 22:26:08 +02:00
Anthony LC
95d3a8cd18 ✏️(project) automatic typo correction
Fix typos in the project.
2025-05-13 16:00:43 +02:00
Anthony LC
4f126ab824 🩺(CI) add lint spell mistakes
We get lot of pull requests about typo.
We add codespell linter in the CI, it will inform
us if we introduce spell mistakes.
2025-05-13 16:00:43 +02:00
Manuel Raynaud
fb90c13dad ♻️(helm) change default customization CM mount path
The mount path used in the backend deployment to mount the customization
file ConfigMap is not the same from the default settings. To avoid extra
configuration we change it to refrlect the default value of
settings.THEME_CUSTOMIZATION_FILE_PATH
2025-05-13 15:19:55 +02:00
Manuel Raynaud
4118d79525 🔧(helm) add celery deployment
We need to configure a deployment dedicated to celery. It is a copy of
the backend one with modification made where it is specific to celery
2025-05-13 15:19:54 +02:00
renovate[bot]
5848f43cb4 ⬆️(dependencies) update python dependencies (#956) 2025-05-12 14:29:04 +00:00
Manuel Raynaud
4b0fd223c8 🐛(back) override AI feature flag in config test
The env.d/development/common file sets
AI_FEATURE_ENABLED=true.
When pytest starts it imports these variables, so
the /api/v1.0/config endpoint returns
AI_FEATURE_ENABLED=True and the test_api_config
assertion fails.

Explicitly overriding AI_FEATURE_ENABLED=False in
test_api_config restores the expected behaviour
and makes the whole test-suite green.

Signed-off-by: ReinforcedKnowledge <reinforced.knowledge@gmail.com>
2025-05-12 15:56:30 +02:00
Manuel Raynaud
31d0733851 🔧(back) configure cache key prefix
We want to change the cache key prefix using an environment variable.
This settings can be changed at every deployment in order to reset to
use a fresh new cache.
2025-05-12 15:56:29 +02:00
Manuel Raynaud
16e20e984c (helm) allow to load custom theme file in a configMap
In order to load a custom theme file with our helm chart, we allow to
load the content of a file into a config map and then use this configmap
as a volume in the backend deployment
2025-05-12 15:56:29 +02:00
Manuel Raynaud
76c28760dc 🔥(back) remove footer endpoint
With the configuration file, the footer endpoint can be removed and will
not be used anymore by the front application.
2025-05-12 15:56:29 +02:00
Manuel Raynaud
d856abb5d8 (back) allow theme customnization using a configuration file
We want to customize the theme by using a configuration file. This
configuration file path can be defined using the settings
THEME_CUSTOMIZATION_FILE_PATH. If this file does not exists or is an
invalid json, an empty json object will be added in the config endpoint.
2025-05-12 15:56:26 +02:00
Manuel Raynaud
25abd964de (backend) manage uploaded file status and call to malware detection
In the attachment_upload method, the status in the file metadata to
processing and the malware_detection backend is called. We check in the
media_auth if the status is ready in order to accept the request.
2025-05-12 15:14:09 +02:00
Manuel Raynaud
a070e1dd87 (backend) configure lasuite.malware_detection module
We want to use the malware_detection module from lasuite library. We add
a new setting MALWARE_DETECTION to configure the backend we want to use.
The callback is also added. It removes the file if it is not safe or
change it's status in the metadata to set it as ready.
2025-05-12 15:13:33 +02:00
Manuel Raynaud
37d9ae8cca (backend) force loading celery shared task in libraries
Library we are using can have celery shared task. We have to make some
modification to load them earlier when the celery app is configure and
when the impress app is loaded.
2025-05-12 15:13:32 +02:00
Zorin95670
29ea6b8ef7 (frontend) Improve test coverage
Improve the test coverage of the "api" modules.

Signed-off-by: Zorin95670 <moittie.vincent@gmail.com>
2025-05-12 14:07:08 +02:00
Zorin95670
a692fa6f39 📝(frontend) Update documentation
Improve and add jsdoc.

Signed-off-by: Zorin95670 <moittie.vincent@gmail.com>
2025-05-12 14:07:08 +02:00
Zorin95670
4d541c5d52 🎨(frontend) Minor refactoring
- improve condition statements
- add "no-var" rule in eslint
- remove some unnecessary variables

Signed-off-by: Zorin95670 <moittie.vincent@gmail.com>
2025-05-12 14:07:08 +02:00
Anthony LC
e5f029ad1d 🚩(frontend) version MIT only
We have some packages that are not MIT compatible,
so if the env var MIT_ONLY is set to true,
we don't build the application with features
that are not MIT compatible.
For the moment, it concerns only the export packages.
2025-05-12 12:00:59 +02:00
ZouicheOmar
bd79f84e07 (frontend) adapt export to callout block
Adapt modal export to include PDF and Docx export
for the callout block.
2025-05-12 09:30:17 +02:00
ZouicheOmar
a070f56339 (frontend) add custom callout block to editor
Add a custom block to the editor, the callout block.
2025-05-12 09:30:17 +02:00
ZouicheOmar
02478acb3f (frontend) add emoji picker component
Add a custom emoji picker component to use in the editor
2025-05-12 09:30:17 +02:00
ZouicheOmar
23aa497db0 (frontend) add emoji-mart packages
We need functionalities and data to implement a custom emoji picker
component, as blocknote's emojipicker component triggers and uses cases
are limited.
add to package.json the following packages:
- "emoji-mart": provides functions and components for
displaying, searching and selecting emojis.
- @emoji-mart-data: offers pre-configured sets of emojis.
- @emoji-mart/react: React Picker component
2025-05-12 09:29:04 +02:00
virgile-dev
d48436bffb 📝(doc) complete contributing policy (#895)
We made mandatory signing commits.
Provided warnings for common gitmoji errors

Signed-off-by: virgile-deville <virgile.deville@beta.gouv.fr>
2025-05-09 20:40:44 +00:00
renovate[bot]
41e4c45934 ⬆️(dependencies) update django to v5.1.9 [SECURITY] (#953) 2025-05-09 16:26:57 +02:00
Anthony LC
6be87ed477 🔖(patch) release 3.2.1
Fixed:
- 🐛(frontend) fix list copy paste
2025-05-07 10:27:39 +02:00
Anthony LC
c96182b3e3 🐛(frontend) fix list copy paste
When we copy paste a list, the pasted
list is not formatted correctly.
By pinning prosemirror-model to 1.25.0,
we avoid this issue.
We added "prosemirror-model" to the
ignored dependencies of Renovate to
avoid to have a bump until the patch
on the Blocknote.js side.
2025-05-07 10:25:48 +02:00
Anthony LC
e79d1d618a ⬆️(dependencies) update js dependencies 2025-05-06 11:51:24 +02:00
renovate[bot]
2691cdd4a2 ⬆️(dependencies) update python dependencies (#934) 2025-05-06 09:35:31 +00:00
Riël Notermans
05a1390bdc 📝(doc) Update env.md add AI_FEATURE_ENABLED
This is false by default.
Without this env setting on true AI will not be available in the
docs application.
The setting was missing in the env options.
2025-05-06 10:54:18 +02:00
Anthony LC
dfe8ae14fe 🐛(docker-compose) unbind the y-provider service with frontend
We cannot add new js dependency locally when we bind the
frontend with the y-provider service. It results in
"EPERM: operation not permitted" when the `node_modules`
has to be updated.
Better to remove the binding, we can add the binding
locally during development on the y-provider.
2025-05-06 10:35:59 +02:00
Anthony LC
74165f6890 🔖(minor) release 3.2.0
Added:
- 🚸(backend) make document search on title
  accent-insensitive
- 🚩 add homepage feature flag
- (settings) Allow configuring PKCE for the SSO
- 🌐(i18n) activate chinese and spanish languages
- 🔧(backend) allow overwriting the data directory
- (backend) add  `django-lasuite` dependency
  (breaking change)
- (frontend) advanced table features

Changed:
- ️(frontend) reduce unblocking time for config
- ♻️(frontend) bind UI with ability access
- ♻️(frontend) use built-in Quote block

Fixed:
- 🐛(nginx) fix 404 when accessing a doc
- 🔒️(drf) disable browsable HTML API renderer
- 🔒(frontend) enhance file download security
- 🐛(backend) race condition create doc
- 🐛(frontend) fix breaklines in custom blocks
2025-05-06 09:33:42 +02:00
Anthony LC
349cbf8eb3 🌐(i18n) update translated strings
Update translated files with new translations
2025-05-06 09:33:42 +02:00
Anthony LC
12ef1a2450 🚩(backend) default enable FRONTEND_HOMEPAGE_FEATURE_ENABLED
We decided to enable the FRONTEND_HOMEPAGE_FEATURE_ENABLED
feature flag by default.
It will not be a breaking change like that.
2025-05-05 11:54:26 +02:00
Anthony LC
9b2f7966f6 🌐(i18n) update translated strings
Update translated files with new translations
2025-05-05 11:17:58 +02:00
Anthony LC
5ad30b404d 🌐(i18n) add PO of new languages
New languages were added to Crowdin.
We import the new translations from Crowdin
to version them in the repository.
2025-05-02 16:25:50 +02:00
Anthony LC
12524f35b7 🌐(i18n) remove chinese language
We're going to make languages ​​configurable
per instance, but until we manage that, we're going
to remove Chinese from the default language list.

- Remove the chinese language from the default language
list.
- Change Spanish to Español
2025-05-02 16:25:50 +02:00
Anthony LC
f8a40cf8cc (frontend) add advanced table features
We added advanced table features to the
table editor, including:
- split / merge cells
- cell background color
- cell text color
- header

We adapted the export and brought some improvements
compare to the previous version.

The export PDF supports colspan (merge horizontally),
but does not support the rowspan (merge vertically)
for now.
2025-04-30 17:22:21 +02:00
Anthony LC
c32fdb67ac (frontend) add @blocknote/code-block
To reduce the bundles size, the highlight syntax
library is not included in blocknote core anymore.
We need to add a separate dependency in order
to have the code block syntax highlight feature.
2025-04-30 17:22:21 +02:00
Anthony LC
7f2a21cdc9 🔥(frontend) remove Quote custom block
Last Blocknote upgrade included a Quote block,
better to use their built-in one.
2025-04-30 17:22:21 +02:00
Anthony LC
4ad917906c ⬆️(dependencies) update js dependencies 2025-04-30 17:22:21 +02:00
Anthony LC
9ca79688c9 ♻️(frontend) bind ui with ability access
Some actions were not available in the frontend
but allowed in the backend, this commit binds the frontend
ui with the ability access coming from the backend.
2025-04-30 17:02:13 +02:00
Manuel Raynaud
7f0eb9117e 🔒️(drf) disable browsable HTML API renderer (#919)
The `BrowsableAPIRenderer` generates a form to test POST/PUT/... actions
and fill the FK fields with unfiltered data. This issue has been spoted
on visio and fixed suitenumerique/meet#508
2025-04-30 16:23:26 +02:00
Quentin BEY
2557c6bc77 (backend) add django-lasuite dependency
Use the OIDC backend from the new library and add settings to setup OIDC
token storage required for later calls to OIDC Resource Servers.
2025-04-29 13:15:43 +02:00
Manuel Raynaud
df173c3ce6 🔧(helmfile) personalize keycloak configuration
The keycloak configuration used in dev environment is too generic and we
can have a conflict with other project that are using the same ingress
domain. Also the namespace was missing in the keycloak extra ConfigMap
leading to creating it in the default namespace.
2025-04-28 21:41:02 +02:00
Anthony LC
b58c991c81 🐛(nginx) fix 404 when accessing a doc
We improve the nginx way to access to a specific
doc.
We stop to wait for a initial attempt that
give a 404. If we see a UUID in the url we will
redirect to the doc/[id] page. Next will then
manage the 404.
2025-04-28 21:41:02 +02:00
Martin Weinelt
96f6aeea60 🔧(backend) Allow overwriting the data directory (#893)
## Purpose

Deployments that don't rely on Docker should be given the option to use
a different data directory.

## Proposal

Allow customization of the `DATA_DIR` through an environment variable of
the same name.

If the environment variable is not set the behaviour remains the same as
before.

Signed-off-by: Martin Weinelt <hexa@darmstadt.ccc.de>
2025-04-28 15:41:28 +00:00
Nathan Panchout
9465f1a6ec 🔒(frontend) enhance file download security (#889)
## Purpose

Added a safety check for URLs in the FileDownloadButton component. Now,
before opening a URL, it verifies if the URL is safe using the isSafeUrl
function.
This prevents potentially unsafe URLs from being opened in a new tab.
2025-04-28 12:50:14 +00:00
virgile-dev
98f11ff8ac 🌐(i18n) add spanish and chinese (#884)
All the spanish and chinese translations are complete on crowdin. We
activate it in django settings and download all translations from
crowdin

Signed-off-by: virgile-deville <virgile.deville@beta.gouv.fr>
2025-04-28 12:36:34 +00:00
renovate[bot]
b29daa2d77 ⬆️(dependencies) update python dependencies (#847)
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [boto3](https://redirect.github.com/boto/boto3) | `==1.37.24` ->
`==1.37.33` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/boto3/1.37.33?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/boto3/1.37.33?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/boto3/1.37.24/1.37.33?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/boto3/1.37.24/1.37.33?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
| [celery](https://docs.celeryq.dev/)
([source](https://redirect.github.com/celery/celery),
[changelog](https://docs.celeryq.dev/en/stable/changelog.html)) |
`==5.5.0` -> `==5.5.1` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/celery/5.5.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/celery/5.5.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/celery/5.5.0/5.5.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/celery/5.5.0/5.5.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
| [django](https://redirect.github.com/django/django)
([changelog](https://docs.djangoproject.com/en/stable/releases/)) |
`==5.1.8` -> `==5.2` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/django/5.2?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/django/5.2?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/django/5.1.8/5.2?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/django/5.1.8/5.2?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
|
[django-extensions](https://redirect.github.com/django-extensions/django-extensions)
([changelog](https://redirect.github.com/django-extensions/django-extensions/blob/main/CHANGELOG.md))
| `==3.2.3` -> `==4.1` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/django-extensions/4.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/django-extensions/4.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/django-extensions/3.2.3/4.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/django-extensions/3.2.3/4.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
|
[django-storages](https://redirect.github.com/jschneier/django-storages)
([changelog](https://redirect.github.com/jschneier/django-storages/blob/master/CHANGELOG.rst))
| `==1.14.5` -> `==1.14.6` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/django-storages/1.14.6?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/django-storages/1.14.6?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/django-storages/1.14.5/1.14.6?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/django-storages/1.14.5/1.14.6?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
|
[drf-spectacular-sidecar](https://redirect.github.com/tfranzel/drf-spectacular-sidecar)
| `==2025.3.1` -> `==2025.4.1` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/drf-spectacular-sidecar/2025.4.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/drf-spectacular-sidecar/2025.4.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/drf-spectacular-sidecar/2025.3.1/2025.4.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/drf-spectacular-sidecar/2025.3.1/2025.4.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
| [ipython](https://redirect.github.com/ipython/ipython) | `==9.0.2` ->
`==9.1.0` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/ipython/9.1.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/ipython/9.1.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/ipython/9.0.2/9.1.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/ipython/9.0.2/9.1.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
| [lxml](https://lxml.de/)
([source](https://redirect.github.com/lxml/lxml),
[changelog](https://git.launchpad.net/lxml/plain/CHANGES.txt)) |
`==5.3.1` -> `==5.3.2` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/lxml/5.3.2?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/lxml/5.3.2?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/lxml/5.3.1/5.3.2?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/lxml/5.3.1/5.3.2?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
| [markdown](https://redirect.github.com/Python-Markdown/markdown)
([changelog](https://python-markdown.github.io/changelog/)) | `==3.7` ->
`==3.8` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/markdown/3.8?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/markdown/3.8?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/markdown/3.7/3.8?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/markdown/3.7/3.8?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
| [openai](https://redirect.github.com/openai/openai-python) |
`==1.70.0` -> `==1.73.0` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/openai/1.73.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/openai/1.73.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/openai/1.70.0/1.73.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/openai/1.70.0/1.73.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
| [pycrdt](https://redirect.github.com/jupyter-server/pycrdt) |
`==0.12.10` -> `==0.12.12` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/pycrdt/0.12.12?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/pycrdt/0.12.12?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/pycrdt/0.12.10/0.12.12?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/pycrdt/0.12.10/0.12.12?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
| [pytest-cov](https://redirect.github.com/pytest-dev/pytest-cov)
([changelog](https://pytest-cov.readthedocs.io/en/latest/changelog.html))
| `==6.0.0` -> `==6.1.1` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/pytest-cov/6.1.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/pytest-cov/6.1.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/pytest-cov/6.0.0/6.1.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/pytest-cov/6.0.0/6.1.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
| [pytest-django](https://redirect.github.com/pytest-dev/pytest-django)
([changelog](https://pytest-django.readthedocs.io/en/latest/changelog.html))
| `==4.10.0` -> `==4.11.1` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/pytest-django/4.11.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/pytest-django/4.11.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/pytest-django/4.10.0/4.11.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/pytest-django/4.10.0/4.11.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
| [ruff](https://docs.astral.sh/ruff)
([source](https://redirect.github.com/astral-sh/ruff),
[changelog](https://redirect.github.com/astral-sh/ruff/blob/main/CHANGELOG.md))
| `==0.11.2` -> `==0.11.5` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/ruff/0.11.5?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/ruff/0.11.5?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/ruff/0.11.2/0.11.5?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/ruff/0.11.2/0.11.5?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
| [sentry-sdk](https://redirect.github.com/getsentry/sentry-python)
([changelog](https://redirect.github.com/getsentry/sentry-python/blob/master/CHANGELOG.md))
| `==2.25.0` -> `==2.25.1` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/sentry-sdk/2.25.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/sentry-sdk/2.25.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/sentry-sdk/2.25.0/2.25.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/sentry-sdk/2.25.0/2.25.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>boto/boto3 (boto3)</summary>

###
[`v1.37.33`](https://redirect.github.com/boto/boto3/blob/HEAD/CHANGELOG.rst#13733)

[Compare
Source](https://redirect.github.com/boto/boto3/compare/1.37.32...1.37.33)

\=======

- api-change:`connect-contact-lens`: \[`botocore`] Making sentiment
optional for ListRealtimeContactAnalysisSegments Response depending on
conversational analytics configuration
- api-change:`datazone`: \[`botocore`] Raise hard limit of authorized
principals per SubscriptionTarget from 10 to 20.
- api-change:`detective`: \[`botocore`] Add support for Detective
DualStack endpoints
- api-change:`dynamodb`: \[`botocore`] Doc only update for API
descriptions.
- api-change:`marketplace-entitlement`: \[`botocore`] Add support for
Marketplace Entitlement Service dual-stack endpoints for CN and GOV
regions
- api-change:`meteringmarketplace`: \[`botocore`] Add support for
Marketplace Metering Service dual-stack endpoints for CN regions
- api-change:`pcs`: \[`botocore`] Changed the minimum length of
clusterIdentifier, computeNodeGroupIdentifier, and queueIdentifier to 3.
- api-change:`verifiedpermissions`: \[`botocore`] Adds deletion
protection support to policy stores. Deletion protection is disabled by
default, can be enabled via the CreatePolicyStore or UpdatePolicyStore
APIs, and is visible in GetPolicyStore.
- bugfix:`download_fileobj`: Fileobj provided in append mode will no
longer allow concurrent writes to preserve data integrity.

###
[`v1.37.32`](https://redirect.github.com/boto/boto3/blob/HEAD/CHANGELOG.rst#13732)

[Compare
Source](https://redirect.github.com/boto/boto3/compare/1.37.31...1.37.32)

\=======

- api-change:`application-autoscaling`: \[`botocore`] Application Auto
Scaling now supports horizontal scaling for Elasticache Memcached
self-designed clusters using target tracking scaling policies and
scheduled scaling.
- api-change:`elasticache`: \[`botocore`] AWS ElastiCache SDK now
supports using MemcachedUpgradeConfig parameter with ModifyCacheCluster
API to enable updating Memcached cache node types. Please refer to
updated AWS ElastiCache public documentation for detailed information on
API usage and implementation.
- api-change:`m2`: \[`botocore`] Introduce three new APIs:
CreateDataSetExportTask, GetDataSetExportTask and
ListDataSetExportHistory. Add support for batch restart for Blu Age
applications.
- api-change:`medialive`: \[`botocore`] AWS Elemental MediaLive /
Features : Add support for CMAF Ingest CaptionLanguageMappings,
TimedMetadataId3 settings, and Link InputResolution.
- api-change:`qbusiness`: \[`botocore`] Adds functionality to
enable/disable a new Q Business Hallucination Reduction feature. If
enabled, Q Business will detect and attempt to remove Hallucinations
from certain Chat requests.
- api-change:`quicksight`: \[`botocore`] Add support to analysis and
sheet level highlighting in QuickSight.

###
[`v1.37.31`](https://redirect.github.com/boto/boto3/blob/HEAD/CHANGELOG.rst#13731)

[Compare
Source](https://redirect.github.com/boto/boto3/compare/1.37.30...1.37.31)

\=======

- api-change:`controlcatalog`: \[`botocore`] The GetControl API now
surfaces a control's Severity, CreateTime, and Identifier for a
control's Implementation. The ListControls API now surfaces a control's
Behavior, Severity, CreateTime, and Identifier for a control's
Implementation.
- api-change:`dynamodb`: \[`botocore`] Documentation update for
secondary indexes and Create_Table.
- api-change:`glue`: \[`botocore`] The TableOptimizer APIs in AWS Glue
now return the DpuHours field in each TableOptimizerRun, providing
clients visibility to the DPU-hours used for billing in managed Apache
Iceberg table compaction optimization.
- api-change:`groundstation`: \[`botocore`] Support tagging Agents and
adjust input field validations
- api-change:`transfer`: \[`botocore`] This launch includes 2
enhancements to SFTP connectors user-experience: 1) Customers can
self-serve concurrent connections setting for their connectors, and 2)
Customers can discover the public host key of remote servers using their
SFTP connectors.

###
[`v1.37.30`](https://redirect.github.com/boto/boto3/blob/HEAD/CHANGELOG.rst#13730)

[Compare
Source](https://redirect.github.com/boto/boto3/compare/1.37.29...1.37.30)

\=======

- api-change:`bedrock-runtime`: \[`botocore`] This release introduces
our latest bedrock runtime API, InvokeModelWithBidirectionalStream. The
API supports both input and output streams and is supported by only
HTTP2.0.
- api-change:`ce`: \[`botocore`] This release supports Pagination traits
on Cost Anomaly Detection APIs.
- api-change:`cost-optimization-hub`: \[`botocore`] This release adds
resource type "MemoryDbReservedInstances" and resource type
"DynamoDbReservedCapacity" to the GetRecommendation,
ListRecommendations, and ListRecommendationSummaries APIs to support new
MemoryDB and DynamoDB RI recommendations.
- api-change:`iotfleetwise`: \[`botocore`] This release adds the option
to update the strategy of state templates already associated to a
vehicle, without the need to remove and re-add them.
- api-change:`securityhub`: \[`botocore`] Documentation updates for AWS
Security Hub.
- api-change:`storagegateway`: \[`botocore`] Added new
ActiveDirectoryStatus value, ListCacheReports paginator, and support for
longer pagination tokens.
- api-change:`taxsettings`: \[`botocore`] Uzbekistan Launch on
TaxSettings Page

###
[`v1.37.29`](https://redirect.github.com/boto/boto3/blob/HEAD/CHANGELOG.rst#13729)

[Compare
Source](https://redirect.github.com/boto/boto3/compare/1.37.28...1.37.29)

\=======

- api-change:`bedrock`: \[`botocore`] New options for how to handle
harmful content detected by Amazon Bedrock Guardrails.
- api-change:`bedrock-runtime`: \[`botocore`] New options for how to
handle harmful content detected by Amazon Bedrock Guardrails.
- api-change:`codebuild`: \[`botocore`] AWS CodeBuild now offers an
enhanced debugging experience.
- api-change:`glue`: \[`botocore`] Add input validations for multiple
Glue APIs
- api-change:`medialive`: \[`botocore`] AWS Elemental MediaLive now
supports SDI inputs to MediaLive Anywhere Channels in workflows that use
AWS SDKs.
- api-change:`personalize`: \[`botocore`] Add support for eventsConfig
for CreateSolution, UpdateSolution, DescribeSolution,
DescribeSolutionVersion. Add support for GetSolutionMetrics to return
weighted NDCG metrics when eventsConfig is enabled for the solution.
- api-change:`transfer`: \[`botocore`] This launch enables customers to
manage contents of their remote directories, by deleting old files or
moving files to archive folders in remote servers once they have been
retrieved. Customers will be able to automate the process using
event-driven architecture.

###
[`v1.37.28`](https://redirect.github.com/boto/boto3/blob/HEAD/CHANGELOG.rst#13728)

[Compare
Source](https://redirect.github.com/boto/boto3/compare/1.37.27...1.37.28)

\=======

- api-change:`ds-data`: \[`botocore`] Doc only update - fixed broken
links.
-   api-change:`ec2`: \[`botocore`] Doc-only updates for Amazon EC2
- api-change:`events`: \[`botocore`] Amazon EventBridge adds support for
customer-managed keys on Archives and validations for two fields:
eventSourceArn and kmsKeyIdentifier.
- api-change:`s3control`: \[`botocore`] Updated max size of Prefixes
parameter of Scope data type.

###
[`v1.37.27`](https://redirect.github.com/boto/boto3/blob/HEAD/CHANGELOG.rst#13727)

[Compare
Source](https://redirect.github.com/boto/boto3/compare/1.37.26...1.37.27)

\=======

- api-change:`bedrock-agent`: \[`botocore`] Added optional
"customMetadataField" for Amazon Aurora knowledge bases, allowing
single-column metadata. Also added optional "textIndexName" for MongoDB
Atlas knowledge bases, enabling hybrid search support.
- api-change:`chime-sdk-voice`: \[`botocore`] Added FOC date as an
attribute of PhoneNumberOrder, added AccessDeniedException as a possible
return type of ValidateE911Address
- api-change:`mailmanager`: \[`botocore`] Add support for Dual_Stack and
PrivateLink types of IngressPoint. For configuration requests, SES Mail
Manager will now accept both IPv4/IPv6 dual-stack endpoints and AWS
PrivateLink VPC endpoints for email receiving.
- api-change:`opensearch`: \[`botocore`] Improve descriptions for
various API commands and data types.
- api-change:`route53`: \[`botocore`] Added us-gov-east-1 and
us-gov-west-1 as valid Latency Based Routing regions for
change-resource-record-sets.
- api-change:`sagemaker`: \[`botocore`] Adds support for i3en, m7i, r7i
instance types for SageMaker Hyperpod
- api-change:`sesv2`: \[`botocore`] This release enables customers to
provide attachments in the SESv2 SendEmail and SendBulkEmail APIs.
- api-change:`transcribe`: \[`botocore`] This Feature Adds Support for
the "zh-HK" Locale for Batch Operations
- enhancement:Eventstream: \[`botocore`] The event streams maximum
payload size is now required to be 24Mb or less.

###
[`v1.37.26`](https://redirect.github.com/boto/boto3/blob/HEAD/CHANGELOG.rst#13726)

[Compare
Source](https://redirect.github.com/boto/boto3/compare/1.37.25...1.37.26)

\=======

- api-change:`application-signals`: \[`botocore`] Application Signals
now supports creating Service Level Objectives on service dependencies.
Users can now create or update SLOs on discovered service dependencies
to monitor their standard application metrics.
- api-change:`codebuild`: \[`botocore`] This release adds support for
environment type WINDOWS_SERVER\_2022\_CONTAINER in ProjectEnvironment
- api-change:`ecr`: \[`botocore`] Fix for customer issues related to AWS
account ID and size limitation for token.
- api-change:`ecs`: \[`botocore`] This is an Amazon ECS documentation
only update to address various tickets.
- api-change:`lexv2-models`: \[`botocore`] Release feature of
errorlogging for lex bot, customer can config this feature in bot
version to generate log for error exception which helps debug
- api-change:`medialive`: \[`botocore`] Added support for SMPTE 2110
inputs when running a channel in a MediaLive Anywhere cluster. This
feature enables ingestion of SMPTE 2110-compliant video, audio, and
ancillary streams by reading SDP files that AWS Elemental MediaLive can
retrieve from a network source.

###
[`v1.37.25`](https://redirect.github.com/boto/boto3/blob/HEAD/CHANGELOG.rst#13725)

[Compare
Source](https://redirect.github.com/boto/boto3/compare/1.37.24...1.37.25)

\=======

- api-change:`cleanrooms`: \[`botocore`] This release adds support for
updating the analytics engine of a collaboration.
- api-change:`sagemaker`: \[`botocore`] Added tagging support for
SageMaker notebook instance lifecycle configurations

</details>

<details>
<summary>celery/celery (celery)</summary>

###
[`v5.5.1`](https://redirect.github.com/celery/celery/blob/HEAD/Changelog.rst#551)

[Compare
Source](https://redirect.github.com/celery/celery/compare/v5.5.0...v5.5.1)

\=====

:release-date: 2025-04-08
:release-by: Tomer Nosrati

What's Changed

```

- Fixed "AttributeError: list object has no attribute strip" with quorum queues and failover brokers (#&#8203;9657)
- Prepare for release: v5.5.1 (#&#8203;9660)

.. _version-5.5.0:
```

</details>

<details>
<summary>django/django (django)</summary>

###
[`v5.2`](https://redirect.github.com/django/django/compare/5.1.8...5.2)

[Compare
Source](https://redirect.github.com/django/django/compare/5.1.8...5.2)

</details>

<details>
<summary>django-extensions/django-extensions
(django-extensions)</summary>

###
[`v4.1`](https://redirect.github.com/django-extensions/django-extensions/blob/HEAD/CHANGELOG.md#41)

[Compare
Source](https://redirect.github.com/django-extensions/django-extensions/compare/4.0...4.1)

Changes:

- Add: show_permissions command
([#&#8203;1920](https://redirect.github.com/django-extensions/django-extensions/issues/1920))
- Improvement: graph_models, style per app
([#&#8203;1848](https://redirect.github.com/django-extensions/django-extensions/issues/1848))
- Fix: JSONField, bulk_update's
([#&#8203;1924](https://redirect.github.com/django-extensions/django-extensions/issues/1924))

###
[`v4.0`](https://redirect.github.com/django-extensions/django-extensions/blob/HEAD/CHANGELOG.md#40)

[Compare
Source](https://redirect.github.com/django-extensions/django-extensions/compare/3.2.3...4.0)

Changes:

-   Improvement: Support for Python 3.12 and 3.13
-   Improvement: Support for Django 5.x
-   Improvement: Switch from setup.{cfg,py} to pyproject.toml
- Improvement: graph_models, Add option to display field choices in
graph_models
([#&#8203;1854](https://redirect.github.com/django-extensions/django-extensions/issues/1854))
- Improvement: graph_models, Add webp support
([#&#8203;1857](https://redirect.github.com/django-extensions/django-extensions/issues/1857))
- Improvement: graph_models, Support for ordering edges on
pydot/dot/graphviz
([#&#8203;1914](https://redirect.github.com/django-extensions/django-extensions/issues/1914))
- Improvement: mail_debug, Update mail_debug command to use aiosmtpd
([#&#8203;1880](https://redirect.github.com/django-extensions/django-extensions/issues/1880))
- Improvement: shell_plus, Improve error message for missing import
([#&#8203;1898](https://redirect.github.com/django-extensions/django-extensions/issues/1898))
- Improvement: reset_db, Add reset_db support for django_tenants
([#&#8203;1855](https://redirect.github.com/django-extensions/django-extensions/issues/1855))
- Improvement: docs, various improvements
([#&#8203;1852](https://redirect.github.com/django-extensions/django-extensions/issues/1852),
[#&#8203;1888](https://redirect.github.com/django-extensions/django-extensions/issues/1888),
[#&#8203;1882](https://redirect.github.com/django-extensions/django-extensions/issues/1882),
[#&#8203;1901](https://redirect.github.com/django-extensions/django-extensions/issues/1901),
[#&#8203;1912](https://redirect.github.com/django-extensions/django-extensions/issues/1912),
[#&#8203;1913](https://redirect.github.com/django-extensions/django-extensions/issues/1913))
- Improvement: jobs, Handle non-package modules when looking for job
definitions
([#&#8203;1887](https://redirect.github.com/django-extensions/django-extensions/issues/1887))
- Improvement: Add django-prometheus DB backends support
([#&#8203;1800](https://redirect.github.com/django-extensions/django-extensions/issues/1800))
- Improvement: Call post_command when the command raises an unhandled
exception
([#&#8203;1837](https://redirect.github.com/django-extensions/django-extensions/issues/1837))
- Fix: sqldiff, do not consider ('serial', 'integer') nor ('bigserial',
'bigint') as a `field-type-differ`
([#&#8203;1867](https://redirect.github.com/django-extensions/django-extensions/issues/1867))
- Fix: shell_plus, Fix start up order and add history
([#&#8203;1869](https://redirect.github.com/django-extensions/django-extensions/issues/1869))
- Remove pipchecker and associated tests
([#&#8203;1906](https://redirect.github.com/django-extensions/django-extensions/issues/1906))
- Following Django's release numbering style more closely (see
https://docs.djangoproject.com/en/5.2/internals/release-process/ )

</details>

<details>
<summary>jschneier/django-storages (django-storages)</summary>

###
[`v1.14.6`](https://redirect.github.com/jschneier/django-storages/compare/1.14.5...1.14.6)

[Compare
Source](https://redirect.github.com/jschneier/django-storages/compare/1.14.5...1.14.6)

</details>

<details>
<summary>tfranzel/drf-spectacular-sidecar
(drf-spectacular-sidecar)</summary>

###
[`v2025.4.1`](https://redirect.github.com/tfranzel/drf-spectacular-sidecar/compare/2025.3.1...2025.4.1)

[Compare
Source](https://redirect.github.com/tfranzel/drf-spectacular-sidecar/compare/2025.3.1...2025.4.1)

</details>

<details>
<summary>ipython/ipython (ipython)</summary>

###
[`v9.1.0`](https://redirect.github.com/ipython/ipython/compare/9.0.2...9.1.0)

[Compare
Source](https://redirect.github.com/ipython/ipython/compare/9.0.2...9.1.0)

</details>

<details>
<summary>lxml/lxml (lxml)</summary>

###
[`v5.3.2`](https://redirect.github.com/lxml/lxml/blob/HEAD/CHANGES.txt#532-2025-04-05)

[Compare
Source](https://redirect.github.com/lxml/lxml/compare/lxml-5.3.1...lxml-5.3.2)

\==================

This release resolves CVE-2025-24928 as described in
https://gitlab.gnome.org/GNOME/libxml2/-/issues/847

## Bugs fixed

-   Binary wheels use libxml2 2.12.10 and libxslt 1.1.42.

- Binary wheels for Windows use a patched libxml2 2.11.9 and libxslt
1.1.39.

</details>

<details>
<summary>Python-Markdown/markdown (markdown)</summary>

###
[`v3.8`](https://redirect.github.com/Python-Markdown/markdown/releases/tag/3.8)

[Compare
Source](https://redirect.github.com/Python-Markdown/markdown/compare/3.7...3.8)

##### Changed

- DRY fix in `abbr` extension by introducing method `create_element`
([#&#8203;1483](https://redirect.github.com/Python-Markdown/markdown/issues/1483)).
-   Clean up test directory by removing some redundant tests and port
    non-redundant cases to the newer test framework.
- Improved performance of the raw HTML post-processor
([#&#8203;1510](https://redirect.github.com/Python-Markdown/markdown/issues/1510)).

##### Fixed

- Backslash Unescape IDs set via `attr_list` on `toc`
([#&#8203;1493](https://redirect.github.com/Python-Markdown/markdown/issues/1493)).
- Ensure `md_in_html` processes content inside "markdown" blocks as they
are
parsed outside of "markdown" blocks to keep things more consistent for
third-party extensions
([#&#8203;1503](https://redirect.github.com/Python-Markdown/markdown/issues/1503)).
- `md_in_html` handle tags within inline code blocks better
([#&#8203;1075](https://redirect.github.com/Python-Markdown/markdown/issues/1075)).
- `md_in_html` fix handling of one-liner block HTML handling
([#&#8203;1074](https://redirect.github.com/Python-Markdown/markdown/issues/1074)).
- Ensure `<center>` is treated like a block-level element
([#&#8203;1481](https://redirect.github.com/Python-Markdown/markdown/issues/1481)).
- Ensure that `abbr` extension respects `AtomicString` and does not
process
perceived abbreviations in these strings
([#&#8203;1512](https://redirect.github.com/Python-Markdown/markdown/issues/1512)).
- Ensure `smarty` extension correctly renders nested closing quotes
([#&#8203;1514](https://redirect.github.com/Python-Markdown/markdown/issues/1514)).

</details>

<details>
<summary>openai/openai-python (openai)</summary>

###
[`v1.73.0`](https://redirect.github.com/openai/openai-python/blob/HEAD/CHANGELOG.md#1730-2025-04-12)

[Compare
Source](https://redirect.github.com/openai/openai-python/compare/v1.72.0...v1.73.0)

Full Changelog:
[v1.72.0...v1.73.0](https://redirect.github.com/openai/openai-python/compare/v1.72.0...v1.73.0)

##### Features

- **api:** manual updates
([a3253dd](a3253dd798))

##### Bug Fixes

- **perf:** optimize some hot paths
([f79d39f](f79d39fbca))
- **perf:** skip traversing types for NotGiven values
([28d220d](28d220de3b))

##### Chores

- **internal:** expand CI branch coverage
([#&#8203;2295](https://redirect.github.com/openai/openai-python/issues/2295))
([0ae783b](0ae783b991))
- **internal:** reduce CI branch coverage
([2fb7d42](2fb7d425cd))
- slight wording improvement in README
([#&#8203;2291](https://redirect.github.com/openai/openai-python/issues/2291))
([e020759](e0207598d1))
- workaround build errors
([4e10c96](4e10c96a48))

###
[`v1.72.0`](https://redirect.github.com/openai/openai-python/blob/HEAD/CHANGELOG.md#1720-2025-04-08)

[Compare
Source](https://redirect.github.com/openai/openai-python/compare/v1.71.0...v1.72.0)

Full Changelog:
[v1.71.0...v1.72.0](https://redirect.github.com/openai/openai-python/compare/v1.71.0...v1.72.0)

##### Features

- **api:** Add evalapi to sdk
([#&#8203;2287](https://redirect.github.com/openai/openai-python/issues/2287))
([35262fc](35262fcef6))

##### Chores

- **internal:** fix examples
([#&#8203;2288](https://redirect.github.com/openai/openai-python/issues/2288))
([39defd6](39defd61e8))
- **internal:** skip broken test
([#&#8203;2289](https://redirect.github.com/openai/openai-python/issues/2289))
([e2c9bce](e2c9bce1f5))
- **internal:** slight transform perf improvement
([#&#8203;2284](https://redirect.github.com/openai/openai-python/issues/2284))
([746174f](746174fae7))
- **tests:** improve enum examples
([#&#8203;2286](https://redirect.github.com/openai/openai-python/issues/2286))
([c9dd81c](c9dd81ce02))

###
[`v1.71.0`](https://redirect.github.com/openai/openai-python/blob/HEAD/CHANGELOG.md#1710-2025-04-07)

[Compare
Source](https://redirect.github.com/openai/openai-python/compare/v1.70.0...v1.71.0)

Full Changelog:
[v1.70.0...v1.71.0](https://redirect.github.com/openai/openai-python/compare/v1.70.0...v1.71.0)

##### Features

- **api:** manual updates
([bf8b4b6](bf8b4b6990))
- **api:** manual updates
([3e37aa3](3e37aa3e15))
- **api:** manual updates
([dba9b65](dba9b656fa))
- **api:** manual updates
([f0c463b](f0c463b478))

##### Chores

- **deps:** allow websockets v15
([#&#8203;2281](https://redirect.github.com/openai/openai-python/issues/2281))
([19c619e](19c619ea95))
- **internal:** only run examples workflow in main repo
([#&#8203;2282](https://redirect.github.com/openai/openai-python/issues/2282))
([c3e0927](c3e0927d3f))
- **internal:** remove trailing character
([#&#8203;2277](https://redirect.github.com/openai/openai-python/issues/2277))
([5a21a2d](5a21a2d799))
- Remove deprecated/unused remote spec feature
([23f76eb](23f76eb0b9))

</details>

<details>
<summary>jupyter-server/pycrdt (pycrdt)</summary>

###
[`v0.12.12`](https://redirect.github.com/jupyter-server/pycrdt/blob/HEAD/CHANGELOG.md#01212)

[Compare
Source](https://redirect.github.com/jupyter-server/pycrdt/compare/0.12.11...0.12.12)

-   Add doc and shared type `events()` async event iterator.
-   Fix deadlock while getting root type from within transaction.

###
[`v0.12.11`](https://redirect.github.com/jupyter-server/pycrdt/blob/HEAD/CHANGELOG.md#01211)

[Compare
Source](https://redirect.github.com/jupyter-server/pycrdt/compare/0.12.10...0.12.11)

-   Upgrade `pyo3` to v0.24.1.

</details>

<details>
<summary>pytest-dev/pytest-cov (pytest-cov)</summary>

###
[`v6.1.1`](https://redirect.github.com/pytest-dev/pytest-cov/blob/HEAD/CHANGELOG.rst#611-2025-04-05)

[Compare
Source](https://redirect.github.com/pytest-dev/pytest-cov/compare/v6.1.0...v6.1.1)

- Fixed breakage that occurs when `--cov-context` and the `no_cover`
marker are used together.

###
[`v6.1.0`](https://redirect.github.com/pytest-dev/pytest-cov/blob/HEAD/CHANGELOG.rst#610-2025-04-01)

[Compare
Source](https://redirect.github.com/pytest-dev/pytest-cov/compare/v6.0.0...v6.1.0)

- Change terminal output to use full width lines for the coverage
header.
Contributed by Tsvika Shapira in `#&#8203;678
<https://github.com/pytest-dev/pytest-cov/pull/678>`\_.
- Removed unnecessary CovFailUnderWarning. Fixes `#&#8203;675
<https://github.com/pytest-dev/pytest-cov/issues/675>`\_.
- Fixed the term report not using the precision specified via
`--cov-precision`.

</details>

<details>
<summary>pytest-dev/pytest-django (pytest-django)</summary>

###
[`v4.11.1`](https://redirect.github.com/pytest-dev/pytest-django/releases/tag/v4.11.1)

[Compare
Source](https://redirect.github.com/pytest-dev/pytest-django/compare/v4.11.0...v4.11.1)


https://github.com/pytest-dev/pytest-django/blob/main/docs/changelog.rst#v4111-2025-04-03

###
[`v4.11.0`](https://redirect.github.com/pytest-dev/pytest-django/releases/tag/v4.11.0)

[Compare
Source](https://redirect.github.com/pytest-dev/pytest-django/compare/v4.10.0...v4.11.0)


https://github.com/pytest-dev/pytest-django/blob/main/docs/changelog.rst#v4110-2025-04-01

</details>

<details>
<summary>astral-sh/ruff (ruff)</summary>

###
[`v0.11.5`](https://redirect.github.com/astral-sh/ruff/blob/HEAD/CHANGELOG.md#0115)

[Compare
Source](https://redirect.github.com/astral-sh/ruff/compare/0.11.4...0.11.5)

##### Preview features

- \[`airflow`] Add missing `AIR302` attribute check
([#&#8203;17115](https://redirect.github.com/astral-sh/ruff/pull/17115))
- \[`airflow`] Expand module path check to individual symbols (`AIR302`)
([#&#8203;17278](https://redirect.github.com/astral-sh/ruff/pull/17278))
- \[`airflow`] Extract `AIR312` from `AIR302` rules (`AIR302`, `AIR312`)
([#&#8203;17152](https://redirect.github.com/astral-sh/ruff/pull/17152))
- \[`airflow`] Update oudated `AIR301`, `AIR302` rules
([#&#8203;17123](https://redirect.github.com/astral-sh/ruff/pull/17123))
- \[syntax-errors] Async comprehension in sync comprehension
([#&#8203;17177](https://redirect.github.com/astral-sh/ruff/pull/17177))
- \[syntax-errors] Check annotations in annotated assignments
([#&#8203;17283](https://redirect.github.com/astral-sh/ruff/pull/17283))
- \[syntax-errors] Extend annotation checks to `await`
([#&#8203;17282](https://redirect.github.com/astral-sh/ruff/pull/17282))

##### Bug fixes

- \[`flake8-pie`] Avoid false positive for multiple assignment with
`auto()` (`PIE796`)
([#&#8203;17274](https://redirect.github.com/astral-sh/ruff/pull/17274))

##### Rule changes

- \[`ruff`] Fix `RUF100` to detect unused file-level `noqa` directives
with specific codes
([#&#8203;17042](https://redirect.github.com/astral-sh/ruff/issues/17042))
([#&#8203;17061](https://redirect.github.com/astral-sh/ruff/pull/17061))
- \[`flake8-pytest-style`] Avoid false positive for legacy form of
`pytest.raises` (`PT011`)
([#&#8203;17231](https://redirect.github.com/astral-sh/ruff/pull/17231))

##### Documentation

- Fix formatting of "See Style Guide" link
([#&#8203;17272](https://redirect.github.com/astral-sh/ruff/pull/17272))

###
[`v0.11.4`](https://redirect.github.com/astral-sh/ruff/blob/HEAD/CHANGELOG.md#0114)

[Compare
Source](https://redirect.github.com/astral-sh/ruff/compare/0.11.3...0.11.4)

##### Preview features

- \[`ruff`] Implement `invalid-rule-code` as `RUF102`
([#&#8203;17138](https://redirect.github.com/astral-sh/ruff/pull/17138))
- \[syntax-errors] Detect duplicate keys in `match` mapping patterns
([#&#8203;17129](https://redirect.github.com/astral-sh/ruff/pull/17129))
- \[syntax-errors] Detect duplicate attributes in `match` class patterns
([#&#8203;17186](https://redirect.github.com/astral-sh/ruff/pull/17186))
- \[syntax-errors] Detect invalid syntax in annotations
([#&#8203;17101](https://redirect.github.com/astral-sh/ruff/pull/17101))

##### Bug fixes

- \[syntax-errors] Fix multiple assignment error for class fields in
`match` patterns
([#&#8203;17184](https://redirect.github.com/astral-sh/ruff/pull/17184))
- Don't skip visiting non-tuple slice in `typing.Annotated` subscripts
([#&#8203;17201](https://redirect.github.com/astral-sh/ruff/pull/17201))

###
[`v0.11.3`](https://redirect.github.com/astral-sh/ruff/blob/HEAD/CHANGELOG.md#0113)

[Compare
Source](https://redirect.github.com/astral-sh/ruff/compare/0.11.2...0.11.3)

##### Preview features

- \[`airflow`] Add more autofixes for `AIR302`
([#&#8203;16876](https://redirect.github.com/astral-sh/ruff/pull/16876),
[#&#8203;16977](https://redirect.github.com/astral-sh/ruff/pull/16977),
[#&#8203;16976](https://redirect.github.com/astral-sh/ruff/pull/16976),
[#&#8203;16965](https://redirect.github.com/astral-sh/ruff/pull/16965))
- \[`airflow`] Move `AIR301` to `AIR002`
([#&#8203;16978](https://redirect.github.com/astral-sh/ruff/pull/16978))
- \[`airflow`] Move `AIR302` to `AIR301` and `AIR303` to `AIR302`
([#&#8203;17151](https://redirect.github.com/astral-sh/ruff/pull/17151))
- \[`flake8-bandit`] Mark `str` and `list[str]` literals as trusted
input (`S603`)
([#&#8203;17136](https://redirect.github.com/astral-sh/ruff/pull/17136))
- \[`ruff`] Support slices in `RUF005`
([#&#8203;17078](https://redirect.github.com/astral-sh/ruff/pull/17078))
- \[syntax-errors] Start detecting compile-time syntax errors
([#&#8203;16106](https://redirect.github.com/astral-sh/ruff/pull/16106))
- \[syntax-errors] Duplicate type parameter names
([#&#8203;16858](https://redirect.github.com/astral-sh/ruff/pull/16858))
- \[syntax-errors] Irrefutable `case` pattern before final case
([#&#8203;16905](https://redirect.github.com/astral-sh/ruff/pull/16905))
- \[syntax-errors] Multiple assignments in `case` pattern
([#&#8203;16957](https://redirect.github.com/astral-sh/ruff/pull/16957))
- \[syntax-errors] Single starred assignment target
([#&#8203;17024](https://redirect.github.com/astral-sh/ruff/pull/17024))
- \[syntax-errors] Starred expressions in `return`, `yield`, and `for`
([#&#8203;17134](https://redirect.github.com/astral-sh/ruff/pull/17134))
- \[syntax-errors] Store to or delete `__debug__`
([#&#8203;16984](https://redirect.github.com/astral-sh/ruff/pull/16984))

##### Bug fixes

- Error instead of `panic!` when running Ruff from a deleted directory
([#&#8203;16903](https://redirect.github.com/astral-sh/ruff/issues/16903))
([#&#8203;17054](https://redirect.github.com/astral-sh/ruff/pull/17054))
- \[syntax-errors] Fix false positive for parenthesized tuple index
([#&#8203;16948](https://redirect.github.com/astral-sh/ruff/pull/16948))

##### CLI

- Check `pyproject.toml` correctly when it is passed via stdin
([#&#8203;16971](https://redirect.github.com/astral-sh/ruff/pull/16971))

##### Configuration

- \[`flake8-import-conventions`] Add import `numpy.typing as npt` to
default `flake8-import-conventions.aliases`
([#&#8203;17133](https://redirect.github.com/astral-sh/ruff/pull/17133))

##### Documentation

- \[`refurb`] Document why `UserDict`, `UserList`, and `UserString` are
preferred over `dict`, `list`, and `str` (`FURB189`)
([#&#8203;16927](https://redirect.github.com/astral-sh/ruff/pull/16927))

</details>

<details>
<summary>getsentry/sentry-python (sentry-sdk)</summary>

###
[`v2.25.1`](https://redirect.github.com/getsentry/sentry-python/blob/HEAD/CHANGELOG.md#2251)

[Compare
Source](https://redirect.github.com/getsentry/sentry-python/compare/2.25.0...2.25.1)

##### Various fixes & improvements

- fix(logs): Add a class which batches groups of logs together.
([#&#8203;4229](https://redirect.github.com/getsentry/sentry-python/issues/4229))
by [@&#8203;colin-sentry](https://redirect.github.com/colin-sentry)
- fix(logs): Use repr instead of json for message and arguments
([#&#8203;4227](https://redirect.github.com/getsentry/sentry-python/issues/4227))
by [@&#8203;colin-sentry](https://redirect.github.com/colin-sentry)
- fix(logs): Debug output from Sentry logs should always be `debug`
level.
([#&#8203;4224](https://redirect.github.com/getsentry/sentry-python/issues/4224))
by [@&#8203;antonpirker](https://redirect.github.com/antonpirker)
- fix(ai): Do not consume anthropic streaming stop
([#&#8203;4232](https://redirect.github.com/getsentry/sentry-python/issues/4232))
by [@&#8203;colin-sentry](https://redirect.github.com/colin-sentry)
- fix(spotlight): Do not spam sentry_sdk.warnings logger w/ Spotlight
([#&#8203;4219](https://redirect.github.com/getsentry/sentry-python/issues/4219))
by [@&#8203;BYK](https://redirect.github.com/BYK)
- fix(docs): fixed code snippet
([#&#8203;4218](https://redirect.github.com/getsentry/sentry-python/issues/4218))
by [@&#8203;antonpirker](https://redirect.github.com/antonpirker)
- build(deps): bump actions/create-github-app-token from 1.11.7 to
1.12.0
([#&#8203;4214](https://redirect.github.com/getsentry/sentry-python/issues/4214))
by [@&#8203;dependabot](https://redirect.github.com/dependabot)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 7am on monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config
help](https://redirect.github.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/suitenumerique/docs).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4yMjcuMyIsInVwZGF0ZWRJblZlciI6IjM5LjIzOC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiLCJub0NoYW5nZUxvZyJdfQ==-->

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Manuel Raynaud <manu@raynaud.io>
2025-04-28 14:05:52 +02:00
Tom Hubrecht
5cdbdbf215 (settings) Allow configuring PKCE for the SSO (#886)
C.f.
https://mozilla-django-oidc.readthedocs.io/en/latest/settings.html#OIDC_USE_PKCE

## Purpose

Add pkce settings

## Proposal
Get the settings from the environment

Signed-off-by: Tom Hubrecht <github@mail.hubrecht.ovh>
2025-04-28 12:54:30 +02:00
Anthony LC
5268699d50 ⬆️(dependencies) update js dependencies 2025-04-23 11:43:50 +02:00
virgile-dev
cdafe6fd33 📝(readme) update xl packages info (#885)
Info message so people fulfill their licencing obligations

Signed-off-by: virgile-deville <virgile.deville@beta.gouv.fr>
2025-04-22 13:57:45 +00:00
Anthony LC
4307b4f433 🐛(backend) race condition create doc
When 2 docs are created almost at the same time,
the second one will fail because the first one.
We get a unicity error on the path key already
used ("impress_document_path_key").
To fix this issue, we will lock the table the
time to create the document, the next query will
wait for the lock to be released.
2025-04-22 11:43:29 +02:00
Anthony LC
3bf33d202a ️(frontend) reduce unblocking time for config
We will serve the config from the cache if available
in waiting for the config to be loaded.
It will remove the loading time for the config except
when the config is not available in the cache.
2025-04-22 11:23:55 +02:00
Anthony LC
101cef7d70 ♻️(frontend) refacto useCunninghamTheme
Refacto useCunninghamTheme, we don't need a function
to have access to the tokens anymore.
2025-04-22 10:38:51 +02:00
Samuel Paccoud - DINUM
419079ac69 🚸(backend) make document search on title accent-insensitive
This should work in both cases:
- search for "vélo" when the document title contains "velo"
- search for "velo" when the document title contains "vélo"
2025-04-17 20:28:14 +02:00
Anthony LC
ecd06560c6 🚚(frontend) Display homepage on /home url
The homepage is now accessible at the /home URL.
Before the homepage was accessible on the /login URL.
We still keep the /login URL for backward compatibility.
2025-04-13 13:25:40 +02:00
Anthony LC
e9ab099ce0 🚩(frontend) integrate homepage feature flag
If the homepage feature flag is enabled,
the homepage will be displayed.
2025-04-13 13:25:40 +02:00
Anthony LC
67b69d05f7 🚩(backend) add homepage feature flag
Add a homepage feature flag that we will
propagate to the frontend.
It will be used to enable or disable the
homepage at runtime.
2025-04-13 13:25:40 +02:00
virgile-deville
f429eb053a 📝(readme) remove preprod account
Having a preprod account using a yopmail account was a security bad practice

Co-authored-by: Samuel Paccoud <sampaccoud@users.noreply.github.com>
2025-04-11 22:29:22 +02:00
Olivier Laurendeau
ad11b7f554 📝(docs) init architecture documentation
- Add docs about architecture
- Add ADR about the CRDT choice
2025-04-10 15:15:15 +02:00
Anthony LC
3d5adad227 🔖(minor) release 3.1.0
Added:
- 🚩(backend) add feature flag for the footer
- 🔧(backend) add view to manage footer json
- (frontend) add custom css style
- 🚩(frontend) conditionally render AI button only
  when feature is enabled

Changed:
- 🚨(frontend) block button when creating doc

Fixed:
- 🐛(back) validate document content in serializer
- 🐛(frontend) fix selection click
  past end of content
2025-04-08 12:41:38 +02:00
Anthony LC
de8e812f2f 🥚(frontend) remove easter egg
Remove the easter egg from the console.
2025-04-07 13:18:04 +02:00
Nathan Panchout
7a1601c682 (frontend) update favicon files and links
- Added new favicon files: favicon-dark.png and favicon.png.
- Updated the _app.tsx file to link to the new favicon files, supporting
both light and dark color schemes.
2025-04-07 13:18:04 +02:00
Nathan Panchout
0537572542 ♻️(frontend) icon component refactoring
- Add variant to IconComponent and remove $isMaterialIcon prop
- Replace all Text component used as icon with the Icon component.
2025-04-07 13:18:04 +02:00
Anthony LC
8aab007ad1 🐛(frontend) do not display firefox modal if not necessary
It is necessary to display the firefox modal only
if the user has something to save.
2025-04-07 13:18:04 +02:00
Anthony LC
cde3de43f7 ♻️(frontend) misc improvements
- add opacity props on Box
- rename to cunningham-style.css
- update illustration-docs-empty.png
- smaller tooltip
2025-04-07 13:18:04 +02:00
Anthony LC
8c0c3c2f44 🐛(frontend) fix selection click past end of content
On Chrome, when we click at the end of a line,
the cursor is placed at the beginning of the line.
We fix this behavior, now the cursor is placed
at the end of the line.
2025-04-07 13:18:04 +02:00
Anthony LC
c11d59c434 🚩(backend) add feature flag for the footer
We added the feature flag `FRONTEND_FOOTER_FEATURE_ENABLED`
to enable or disable the footer in the frontend.
2025-04-04 15:44:38 +02:00
Manuel Raynaud
8836109945 ♻️(back) reset cache after every test
We move the cache reset in the global conf test to not have to think
about reseting the cache when we implement test.
2025-04-04 15:44:38 +02:00
Anthony LC
ba136ff82f 🔧(backend) add view to manage footer json
We added the `FRONTEND_URL_JSON_FOOTER` environment
variable. It will give the possibility to generate
your own footer content in the frontend.
If the variable is not set, the footer will not
be displayed.
2025-04-04 15:44:38 +02:00
Anthony LC
96d9d1a184 🔊(y-provider) improve and add logs
We have somes entries with "No cookies", we
add more logs to understand why we have this case.
We add the datetime in front of each entries as
well.
2025-04-03 16:27:40 +02:00
Manuel Raynaud
771ffdc7cc 🔥(y-provider) remove npm in docker image
We use yarn and not npm, we remove npm because it has a dependencie with
cross-spawn which has a CVE.
2025-04-03 10:41:55 +02:00
Manuel Raynaud
82eba1e8ea 🔥(ci) force ci to fails if trivy fails
If trivy fails we must stop the CI to avoid publishing images with
security issues.
2025-04-03 10:41:55 +02:00
renovate[bot]
8c42599d0f ⬆️(dependencies) update django to v5.1.8 [SECURITY] 2025-04-03 10:28:12 +02:00
renovate[bot]
8620cf4857 ⬆️(dependencies) update next to v15.2.4 [SECURITY] 2025-04-03 07:02:20 +02:00
Baptiste Fontaine
2a7da73248 📝(docs) fix grammar and typos
Fix grammar and typos in docs/installation.md file.
2025-04-02 11:53:56 +02:00
Manuel Raynaud
e8e9922832 (back) remove url-normalize dependency
We have the dependency url-normalize installed but we don't use it in
our codebase.
2025-04-01 09:51:29 +02:00
renovate[bot]
2da4ce4570 ⬆️(dependencies) update python dependencies 2025-04-01 09:51:29 +02:00
Anthony LC
50b90f9ae7 (e2e) fix some flaky tests
Some tests were flaky, causing them to fail
intermittently. This commit aims to address
this issue.
2025-03-31 12:34:04 +02:00
Anthony LC
65ddf7fbe8 📝(docs) add documentation about runtime theming
Add a documentation page about runtime theming.
It explains how to use the theming system
and provide a example.
2025-03-31 12:34:04 +02:00
Anthony LC
d3a7ee74b3 💄(frontend) add classname to components
To be easily customized, we added a classname
to most of the components.
2025-03-31 12:34:04 +02:00
Anthony LC
65e450c6cc (frontend) add custom css style
From the config, we can add custom css style
to the app.
2025-03-31 12:34:04 +02:00
Anthony LC
725cae5470 🔧(backend) add FRONTEND_CSS_URL env var
We added the `FRONTEND_CSS_URL` environment
variable. It will give the possibility to add a
css layer at runtime.
2025-03-31 12:34:04 +02:00
Anthony LC
3881930e82 🚨(frontend) block button when creating doc
When the user clicks on the button to create a new doc,
the button is disabled to prevent multiple clicks.
Multiple clicks on the button could create multiple docs
and create a error about duplicated paths.
2025-03-31 11:29:28 +02:00
Anthony LC
910686293c ️(frontend) improve heading store
Reset headings only when the headers are not
equal to the previous ones. It will prevent
unnecessary rerenders.
2025-03-31 11:29:28 +02:00
Anthony LC
7e7c9ac4c5 ♻️(frontend) replace useModal hook
useModal hook does not use useCallback for its
methods that creates useless rerenders.
2025-03-31 11:29:28 +02:00
Anthony LC
d5d2cfab8e ♻️(frontend) improve useSaveDoc hook
- add tests for useSaveDoc hook
- simplify hooks states
- reduce useEffect calls
2025-03-31 11:29:28 +02:00
Matthias
f2ed8e0ea1 🐛(frontend) conditionally render AI button in toolbar
Added a feature flag check to ensure the AIGroupButton is only rendered
when AI_FEATURE_ENABLED is explicitly set to "true". This prevents the
AI button from appearing when the feature is not configured or disabled.

Fixes #782

Signed-off-by: Matthias <matthias@universum.com>
2025-03-31 11:04:00 +02:00
Manuel Raynaud
fbe8a26dba 🐛(back) validate document content in serializer
We recently extract images url in the content. For this, we assume that
the document content is always in base64. We enforce this assumption by
checking if it's a valide base64 in the serializer.
2025-03-29 19:08:39 +01:00
Berry den Hartog
3e974be9f4 📝(docs) describe environmental options for docs backend (#821)
Signed-off-by: 
Berry den Hartog <38954346+berrydenhartog@users.noreply.github.com>
2025-03-28 17:15:35 +00:00
Bastien Guerry
10f9d25920 📄(legal) add warning about legal compliance
We need to double-check our legal constraints regarding the use of XL
packages within Docs. In the meantime, sends a message to potential
reusers.
2025-03-28 17:15:16 +01:00
Jacques ROUSSEL
4178693e63 🐛(ci) use github action for argocd webhook notification
In order to refactor this notification between alls projetcs, we choose
to use a custom github action
2025-03-28 16:42:45 +01:00
Anthony LC
53be6de5f8 🔖(major) release 3.0.0
Added:
- 📄(legal) Require contributors to sign a DCO

Changed:
- ♻️(frontend) Integrate UI kit
- 🏗️(y-provider) manage auth in y-provider app

Fixed:
- 🐛(backend) compute ancestor_links in get_abilities
  if needed
- 🔒️(back) restrict access to document accesses
2025-03-28 15:32:08 +01:00
Anthony LC
4ff90abdee 🐛(service-worker) force reload new service worker
When multiple tabs are open, the new service worker
can stay in the "waiting" state and not be activated
until the other tabs with the old service worker
are closed.
We fix this by forcing the other tabs to reload
the page when a new service worker is detected.
All tabs will then be reloaded and the new service
worker will be activated.
2025-03-28 15:32:08 +01:00
Anthony LC
544dd00c16 🔧(helm) adapt setting helm dev file
The way that collaboration server authentifies the user
has changed. We adapt the configuration to the new
way of doing it, by removing the nginx auth url,
and by adding COLLABORATION_BACKEND_BASE_URL
setting.
2025-03-28 15:32:08 +01:00
Anthony LC
a3cd4c51ea 🩹(frontend) fine tunning for v3.0.0
- fix width select export
2025-03-28 15:32:08 +01:00
Manuel Raynaud
7e1eed3abd (y-provider) check hocuspocus documentName validity
We only use uuid v4 as hocuspocus dicument name. To be sure nothing else
is used we check that the documentName is a valid uuid version 4.
2025-03-27 18:42:04 +01:00
Manuel Raynaud
8bee476b5b 🔥(back) remove collaboration-auth endpoint
We don't need anymore the collaboration-auth endpoint. Every code
related to it is removed.
2025-03-27 18:42:04 +01:00
Manuel Raynaud
e86919fb9a 🏗️(y-provider) manage auth in y-provider app
The way to connect to the hocuspocus server needs to be proxified in
nginx to query a dedicated route in the django application and then
follow the request to the express server with the additionnal headers.
The auth can be done in the express server by querying the backend on
the document retrieve endpoint. If the response status code is 200, the
user has access to the document, otherwise it is not the case. Then we
can check the abilities to determine what the user can do or not.
2025-03-27 18:42:04 +01:00
Manuel Raynaud
a5b9169eb6 ♻️(back) replace Ypy by pycrdt
Ypy is deprecated and unmaintained. We have problem with parsing
existing documents. We replace it by pycrdt, library actively maintained
and without the issues we have with Ypy.
2025-03-27 18:27:04 +01:00
Manuel Raynaud
c0dfb4b6b3 ♻️(back) remove filtering on logging handler
Level filtering was used on the logging console handler. We remove as it
is not necessary to have it.
2025-03-27 18:27:04 +01:00
Manuel Raynaud
be051ad7d2 🐛(ci) use sha256 to sign argocd webhook call
The argocd webhook call needs now to use sha256 digest now to sign
2025-03-27 18:27:04 +01:00
Manuel Raynaud
a4452784e1 🔒️(back) restrict accesss to document accesses
Every user having an access to a document, no matter its role have
access to the entire accesses list with all the user details. Only
owner or admin should be able to have the entire list, for the other
roles, they have access to the list containing only owner and
administrator with less information on the username. The email and its
id is removed
2025-03-26 10:40:53 +01:00
Quentin BEY
2929e98260 ♻️(documents) inherit manager from queryset
During a code review, I saw we are overriding the MP_NodeManager and
redefine the queryset filters:

- The MP_NodeManager sorts the queryset by `path` by default and it's
  not done on our side, is it on purpose?
- The fact we need to redefine `readable_per_se` as a boilerplate is
  surprising.

I suggest we use the Django mechanism to generate the manager from the
queryset.
2025-03-24 15:04:50 +01:00
Manuel Raynaud
a1914c6259 🐛(backend) compute ancestor_links in get_abilities if needed
The refactor made in the tree view caching the ancestors_links to not
compute them again in the document.get_abilities method lead to a bug.
If the get_abilities method is called without ancestors_links, then they
are computed on all the ancestors but not from the highest readable
ancestor for the current user. We have to compute them with this
constraint.
2025-03-24 14:04:46 +01:00
Samuel Paccoud - DINUM
c882f1386c ♻️(backend) remove lazy from languages field on User model
The idea behind wrapping choices in `lazy` function was to allow
overriding the list of languages in tests with `override_settings`.
This was causin makemigrations to keep on including the field in
migrations when it is not needed. Since we finally don't override
the LANGUAGES setting in tests, we can remove it to fix the problem.
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
c02f19a2cd (backend) extract attachment keys from updated content for access
We can't prevent document editors from copy/pasting content to from one
document to another. The problem is that copying content, will copy the
urls pointing to attachments but if we don't do anything, the reader of
the document to which the content is being pasted, may not be allowed to
access the attachment files from the original document.

Using the work from the previous commit, we can grant access to the readers
of the target document by extracting the attachment keys from the content and
adding themto the target document's "attachments" field. Before doing this,
we check that the current user can indeed access the attachment files extracted
from the content and that they are allowed to edit the current document.
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
34a208a80d (backend) add duplicate action to the document API endpoint
We took this opportunity to refactor the way access is controlled on
media attachments. We now add the media key to a list on the document
instance each time a media is uploaded to a document. This list is
passed along when a document is duplicated, allowing us to grant
access to readers on the new document, even if they don't have or
lost access to the original document.

We also propose an option to reproduce the same access rights on the
duplicate document as what was in place on the original document.
This can be requested by passing the "with_accesses=true" option in
the query string.

The tricky point is that we need to extract attachment keys from the
existing documents and set them on the new "attachments" field that is
now used to track access rights on media files.
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
6976bb7c78 (backend) fix migration test using model factory
Migration tests should not import and use factories or models
directly from the code because they would not be in sync with
the database in the state that each state needs to test it.

Instead the migrator object passed as argument allows us to
retrieve a minimal version of the models in sync with the state
of the database that we are testing. What we get is a minimal
model and we need to simulate all the methods that we could have
on the real model and that are needed for testing.
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
621393165f (backend) add missing test on media-auth and collaboration-auth
These methods were involved in a bug that was fixed without first
evidencing the error in a test:
https://github.com/suitenumerique/docs/pull/556

Fixes https://github.com/suitenumerique/docs/issues/567
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
3e9b530985 (backend) add missing tests for collaboration auth
Tests were forgotten. While writing the tests, I fixed
a few edge cases like the possibility to connect to the
collaboration server for an anonymous user.
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
54f9b3963e ♻️(backend) refactor media_auth and collaboration_auth for flexibility
These 2 actions had factorized code but a few iterations lead to
spaghetti code where factorized code includes "if" clauses.

Refactor abstractions so that code factorization really works.
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
710bbf512c (backend) add util to extract text from Ydoc content
Documents content is stored in the Ydoc format. We need a util
to extract it as xml/text.
2025-03-24 10:43:45 +01:00
Jacques ROUSSEL
747ca70186 🐛(ci) fix Tilt resources dependencies
The Tilt stack was not starting properly due to dependency issues. We
need to wait for PostgreSQL to be running before starting the migration.
2025-03-24 09:33:15 +01:00
renovate[bot]
9374495fda ⬆️(dependencies) update next to v15.2.3 [SECURITY] 2025-03-24 09:18:33 +01:00
Bastien Guerry
ef7cc67387 📄(legal) Require contributors to sign a DCO
Contributors are required to sign off their commits: this confirms
that they have read and accepted https://developercertificate.org.
2025-03-23 09:57:35 +01:00
Sylvain Zimmer
a8529e434a 🐛(media) fix compatibility with Scaleway Object Storage
Some providers with S3-compatible APIs have slightly different
implementations. In this case, Scaleway didn't accept version_id=""
and has a different version ID scheme. This was tested successfully
and should remain compatible with any other provider.
2025-03-22 18:00:43 +01:00
Manuel Raynaud
f8203a1766 🚨(back) lint code with ruff 0.11.2
New Ruff rule (C420) detects code that should be linted. We apply this
new rule on our code.
2025-03-22 10:28:48 +01:00
renovate[bot]
ce8b98e256 ⬆️(dependencies) update python dependencies 2025-03-22 10:28:48 +01:00
Anthony LC
4243519eee 🔥(frontend) remove Marianne font
Marianne font is now part of the UI kit.
We can remove it from the project.
2025-03-21 17:49:06 +01:00
Nathan Panchout
1abf529891 (frontend) refactor and theme token update
The configuration file has been simplified by importing configurations
from @gouvfr-lasuite/ui-kit . Colors and components have been updated to
reflect the new values. Additionally, adjustments have been made to
global styles, including the addition of styles for Material icons. Form
components have also been modified to incorporate the new style
properties.
2025-03-21 17:49:06 +01:00
Nathan Panchout
69ca4af539 (frontend) updated dependencies and added new packages
Added several new dependencies to the `package.json` file, including
`@dnd-kit/core`, `@dnd-kit/modifiers`, `@fontsource/material-icons`, and
`@gouvfr-lasuite/ui-kit`.
2025-03-21 17:49:06 +01:00
Anthony LC
14b2adedfb 🔖(minor) release 2.6.0
Added:
- 📝(doc) add publiccode.yml

Changed
- 🚸(frontend) ctrl+k modal not when editor is focused

Fixed:
- 🐛(back) allow only images to be used with
  the cors-proxy
- 🐛(backend) stop returning inactive users
  on the list endpoint
- 🔒️(backend) require at least 5 characters
  to search for users
- 🔒️(back) throttle user list endpoint
- 🔒️(back) remove pagination and limit to
   5 for user list endpoint
2025-03-21 17:07:26 +01:00
Anthony LC
a7edb382a7 🩹(frontent) change selector to block cmd+k
Multiple ctrl+k could open the search modal, we
change the selector, now if the toolbar is displayed
we don't open the search modal.
2025-03-21 17:07:26 +01:00
Anthony LC
fb5400c26b ️(frontend) search users with at least 5 characters
We now only search for users when the query
is at least 5 characters long.
2025-03-21 15:44:09 +01:00
Manuel Raynaud
8473facbee 🔒️(back) throttle user list endpoint
The user list endpoint is throttle to avoid users discovery. The
throttle is set to 500 requests per day. This can be changed using the
settings API_USERS_LIST_THROTTLE_RATE.
2025-03-21 15:44:09 +01:00
Anthony LC
5db446e8a8 🏷️(frontend) adapt type for user search
The response from the user request is now an
array of users, we don't paginate anymore.
We adapt the types to reflect this.
2025-03-21 15:44:09 +01:00
Manuel Raynaud
34dfb3fd66 🔒️(back) remove pagination and limit to 5 for user list endpoint
The user list endpoint does not use anymore a pagination, the results is
directly return in a list and the max results returned is limited to 5.
In order to modify this limit the settings API_USERS_LIST_LIMIT is
used.
2025-03-21 15:44:09 +01:00
Samuel Paccoud - DINUM
f9a91eda2d 🐛(backend) stop returning inactive users on the list endpoint
inactive users should not be returned as we don't want users to be
able to share new documents with them.
2025-03-21 15:44:09 +01:00
Samuel Paccoud - DINUM
eba926dea4 🔒️(backend) require at least 5 characters to search for users
Listing users is made a little to easy for authenticated users.
2025-03-21 15:44:09 +01:00
Anthony LC
3839a2e8b1 💄(frontend) improve contrast of Beta icon
The colors of the Beta icon were not contrasted
enough. This was posing an accessibility issue.
We now use a more contrasted color.
2025-03-21 09:22:42 +01:00
Anthony LC
a88d62e07d 🌐(frontend) make Docs title translatable
The title of the docs page was not translatable.
We now use the `t` function to translate the title.
2025-03-21 09:22:42 +01:00
Paul Mustière
b61a7a4961 📝(docs) fix typo
Correct language to not be past tense
2025-03-21 06:38:27 +01:00
Anthony LC
20d32ecc4e 🚸(frontend) ctrl+k modal not when editor is focused
ctrl+k interaction was as well used in the editor.
So if the user has a focus on the editor, we don't
open the searchmodal.
2025-03-20 17:43:32 +01:00
Manuel Raynaud
313acf4f78 🐛(back) allow only images to be used with the cors-proxy
The cors-proxy endpoint allowed to use every type of files and to
execute it in the browser. We limit the scope only to images and
Content-Security-Policy and Content-Disposition headers are also added
to not allow script execution that can be present in a SVG file.
2025-03-20 16:10:47 +01:00
Bastien
3a6105cc7e 📝(doc) add publiccode.yml (#770)
publiccode.yml is a standard for describing Free Software projects,
similar to other initiatives such as https://codemeta.github.io.

It is particularly suitable for describing projects funded by public
administrations. See https://github.com/publiccodeyml/publiccode.yml
2025-03-19 21:28:32 +01:00
Anthony LC
bbe17156be 🔖(minor) release 2.5.0
Added:
- 📝(doc) Added GNU Make link to README
- (frontend) add pinning on doc detail
- 🚩(frontend) feature flag analytic on copy as html
- (frontend) Custom block divider with export
- 🌐(i18n) activate dutch language

Changed:
- 🧑‍💻(frontend) change literal section open source
- ♻️(frontend) replace cors proxy for export
- 🚨(gitlint) Allow uppercase in commit messages

Fixed:
- 🐛(frontend) SVG export
- 🐛(frontend) remove scroll listener table content
- 🔒️(back) restrict access to favorite_list endpoint
- 🐛(backend) refactor to fix filtering on children
    and descendants views
- 🐛(action) fix notify-argocd workflow
- 🚨(helm) fix helmfile lint
- 🚚(frontend) redirect to 401 page when 401 error
2025-03-19 14:11:47 +01:00
Anthony LC
51cc26b916 🐛(frontend) improve svg export to be less pixelized
Some SVGs were pixelized in the exported files.
We now add the wanted size to the svg conversion to
make sure the images are exported with the correct size
and so less pixelized.
2025-03-19 14:11:47 +01:00
Anthony LC
cab8ef51df 🐛(frontend) unmount components Analytics
`useAnalytics` hooks was dispatching methods that
caused children components to be unmounted.
By declaring the methods out of the hook, we can
prevent the components from being unmounted.
2025-03-18 14:53:09 +01:00
Anthony LC
6627518017 🚚(frontend) redirect to 401 page when 401 error
Users could still be able to edit a document if the
session was expired. It could give the feeling that the
document was not saved.
If during a mutation request (POST, PUT, DELETE),
the server returns a 401 error,
the user is redirected to the 401 page.
2025-03-18 14:53:09 +01:00
Pedro Manse
12c18bc4e9 📝(README) Add link to GNU Make
Just like docker-compose, create link to the software's site on it's
first mention.

📝(Changelog) Added entry

📝(Changelog) Added pull request id
2025-03-18 11:07:22 +01:00
Anthony LC
aff330eb5b 🚨(gitlint) Allow uppercase in commit messages
Many developers use uppercase as the first letter
in their commit messages, it creates an error.
We will allow uppercase in commit messages to
lower frustration when committing.
2025-03-18 10:24:08 +01:00
Cameron King
bcdaedba9b 🐛(backend) add user/db to pg healthchecks
Adds PostgreSQL user and database names to the docker-compose.yaml healthchecks.
This resolves an error that appears in the logs, where 'root' is used by
default.
2025-03-18 09:41:27 +01:00
Manuel Raynaud
799814e3e3 🌐(i18n) activate dutch language
All the dutch translations are complete on crowdin. We activate it in
the django settings and download all translations from crowdin
2025-03-18 09:27:13 +01:00
virgile-dev
02c9b2ea2e 🐛(readme) fix preprod link to redirect to homepage (#747)
The current link redirects to a 404. New link redirect to homepage.
2025-03-17 16:02:45 +00:00
Manuel Raynaud
eb23aefd55 ♻️(back) use same base route path for swagger
Swaggers urls where not using the same base route path /api/v1.0, we
prepend it to have the same path everywhere. Moreover, a double slash
was used for swagger and redoc dashboard.
2025-03-17 15:02:34 +01:00
Manuel Raynaud
0c49019490 🚨(helm) fix helmfile lint
Latest release of helmfile is applying the change related before as a
warning. Environnements must be before releases but not in the same
document of repositories.
2025-03-17 14:40:55 +01:00
Anthony LC
170dbe07bb ⬆️(frontend) bump @babel/runtime /src/frontend
Bumps @babel/runtime from 7.26.7 to 7.26.10.
2025-03-17 13:50:20 +01:00
Manuel Raynaud
70136f2415 🐛(action) fix notify-argocd workflow
The notify-argocd workflow was not working correctly. The html_url sent
to argocd was not the good one anymore.
2025-03-17 12:09:18 +01:00
Anthony LC
2a8fc97f2f ⬆️(frontend) bump @babel/helpers in /src/frontend
Bumps @babel/helpers from 7.26.7 to 7.26.10.
2025-03-17 11:50:22 +01:00
Anthony LC
9570701bc3 ⬆️(frontend) bump @babel/runtime /src/mail
Bumps @babel/runtime from 7.26.0 to 7.26.10.
2025-03-17 11:36:04 +01:00
Anthony LC
4b28b3c23b (frontend) add pinning on doc detail
Add pinning button on doc detail page.
2025-03-17 11:16:50 +01:00
Anthony LC
f26fc43df0 🔥(frontend) remove DocTagPublic component
DocTagPublic component was removed because
it was not used.
2025-03-17 11:16:50 +01:00
Anthony LC
05a6818439 🧑‍💻(e2e) display more information when auth fails
When the auth fails, it was quite obscure to
understand what was going on.
We now take a screenshot of the page and display
the console logs.
2025-03-17 09:30:19 +01:00
Anthony LC
8056fd7d66 🚚(frontend) add import path for features/docs tsconfig
To simplify imports, we add the import path @/docs
to target ./src/features/docs/.
We changed all imports to use this path.
2025-03-17 09:30:19 +01:00
Samuel Paccoud - DINUM
c85224af42 📝(README) remove AGPL mention
This mention was confusing. We are only using min.io for development
purposes and this has nothing to do with the project's licence.
2025-03-16 19:00:14 +01:00
Anthony LC
70f1b6a8e8 🚩(frontend) add feature flag on "Copy as HTML"
As a blue print, we add a feature flag on
"Copy as HTML" button in the doc toolbox.
This feature flag is controlled by the `CopyAsHtml`
feature flag.

Be aware:
- if the feature flag is disabled, the button
will be shown
- if the feature flag is enabled and send true,
the button will be shown
- if the feature flag is enabled and send false,
the button will be hidden
2025-03-14 16:26:12 +01:00
Anthony LC
0f07fdcb65 📈(frontend) get user analytics
We will identify users by their email address.
This will help us understand how users interact
with the platform and improve the user experience.
2025-03-14 16:26:12 +01:00
Anthony LC
2e13dfb9bc 📈(frontend) abstract analytics classes
Add abstract classes for analytics services.
We will be able to add easily any analytic
services.
Our first analytic service usecase is Posthog.
2025-03-14 16:26:12 +01:00
renovate[bot]
a026435eb7 ⬆️(dependencies) update canvg to v4.0.3 [SECURITY] 2025-03-14 15:29:27 +01:00
Anthony LC
7007d56c38 🐛(frontend) fix svg not rendering export dox
The svg was not rendering in the dox export.
We overwrite the default mapping to convert the
svg to png before rendering.
The images could be out of the page as well,
we fixed this issue by adding a maxWidth to the image.
2025-03-14 15:13:51 +01:00
Anthony LC
0405e6a3f6 🐛(frontend) fix svg not rendering export pdf
The svg was not rendering in the pdf export.
We overwrite the default mapping to convert the
svg to png before rendering.
The images could be out of the page as well,
we fixed this issue by adding a maxWidth to the image.
2025-03-14 15:13:51 +01:00
Anthony LC
cb8bd4b937 🐛(frontend) fix flakiness e2e tests title doc
- fix multiple states title
- wait for stabilize network after create
- fix test other chromium browsers
- improve grid delete test
2025-03-14 14:49:17 +01:00
Anthony LC
4316b4e67d (frontend) adapt export to divider block
We have a new block type, the divider block.
We have to adapt the export to handle this
new block type.
2025-03-14 10:39:07 +01:00
Anthony LC
534085439f (frontend) add divider blocks to the editor
Add a custom block to add a divider in the editor.
2025-03-14 10:39:07 +01:00
Anthony LC
da02d3d756 🎨(frontend) use blockquote tag for quote block
Use the blockquote tag for quote block instead of
a paragraph tag.
2025-03-13 16:38:33 +01:00
Anthony LC
87960d3773 (AI) add emojify action to ai transform
The emojify action add emojis to the important
parts of the text.
2025-03-13 16:38:33 +01:00
Anthony LC
e0af6d36e1 (AI) add beautify action to ai transform
The beautify action add emojis to the important
parts of the text and add formatting to the text
to make it more readable.
2025-03-13 16:38:33 +01:00
Anthony LC
cbf9091d1c ️(AI) improve formating of ai translation
The ai translation were quite lossy about formatting.
Colors, background, breaklines, table sizes were
lost in the translation.
We improve the AI translation request to keep
the formatting as close as possible by using
html instead of markdown.
2025-03-13 16:38:33 +01:00
Anthony LC
9176328200 ♻️(frontend) replace cors proxy for export
We were using the cors proxy of Blocknote.js
to export the document. Now we use our own proxy
to avoid CORS issues.
2025-03-13 12:39:32 +01:00
Anthony LC
6efc2377fe (back) create a cors proxy fetching docs external resources
When exporting a document in PDF and if the doc contains external
resources, we want to fetch them using a proxy bypassing CORS
restrictions. To ensure this endpoint is not used for something else
than fetching urls contains in the doc, we use access control and check
if the url really exists in the document.
2025-03-13 12:39:32 +01:00
Anthony LC
1c02b0ad8e 📝(frontend) change literal section open source
Change literal section open source.
2025-03-12 09:53:01 +01:00
Manuel Raynaud
007854a877 ️(back) use redis as session backend in developement
We want to persist the session during development. Otherwise the session
is reset everytime the server is restart. This behavior make developing
bot a front and back feature a nigthmare, we spend our time login again
and again
2025-03-12 08:28:20 +01:00
Anthony LC
57cead448d 🏷️(frontend) adapt new doc type nb_accesses_direct
We renamed the `nb_accesses` field to `nb_accesses_direct`
and added a new `nb_accesses_ancestors` field.
We adapt the frontend to use the new fields.
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
f20d256cd1 🐛(backend) fix numchild when soft deleting/restoring a document
The numchild attribute must be incremented/decremented manually
when we soft delete a document if we want it to remain accurate,
which is important to display the tree structure in the frontend.
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
76c01df3ae (backend) add number of direct accesses related to a document
The "nb_accesses" field was displaying the number of access instances
related to a document or any of its ancestors. Some features on the
frontend require to know how many of these access instances are related
to the document directly.
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
20315e9b60 (backend) limit link reach/role select options depending on ancestors
If a document already gets a link reach/role inheriting from one of its
ancestors, we should not propose setting link reach/role on the
document that would be more restrictive than what we inherited from
ancestors.
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
2203d49a52 (backend) add new "descendants" action to document API endpoint
We want to be able to make a search query inside a hierchical document.
It's elegant to do it as a document detail action so that we benefit
from access control.
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
56aa69f56a ♻️(backend) refactor list view to allow filtering other views
the "filter_queryset" method is called in the middle of the
"get_object" method. We use the "get_object" in actions like
"children", "tree", etc. which start by calling "get_object"
but return lists of documents.

We would like to apply filters to these views but the it didn't
work because the "get_object" method was also impacted by the
filters...

In a future PR, we should take control of the "get_object" method
and decouple all this. We need a quick solution to allow releasing
the hierchical documents feature in the frontend.
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
0aabf26694 (backend) add "tree" action on document API endpoint
We want to display the tree structure to which a document belongs
on the left side panel of its detail view. For this, we need an
endpoint to retrieve the list view of the document's ancestors
opened.

By opened, we mean that when display the document, we also need to
display its siblings. When displaying the parent of the current
document, we also need to display the siblings of the parent...
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
fcf8b38021 (backend) allow forcing page size within limits
We want to be able to increase the page size with by passing
the query string parameter "page_size".
2025-03-11 09:32:48 +01:00
renovate[bot]
757d7f35cd ⬆️(dependencies) update django to v5.1.7 [SECURITY] 2025-03-10 10:19:56 +01:00
Anthony LC
fdc49dc002 🐛(frontend) remove scroll listener table content
During a useEffect cleaning, the selector was
not the correct one. The debounce was not being
removed correctly neither.
2025-03-10 10:06:59 +01:00
Anthony LC
197ba47f73 ⬆️(frontend) bump cunningham to 3.0.0
Last bump to react 19 was a breaking change with
the previous version of Cunnigham, so we need to
update cunningham to 3.0.0 to be compatible with it.
We can now remove Cunnigham from the list of ignored
dependencies in the renovate.json file.
2025-03-10 09:26:19 +01:00
Anthony LC
d5997ba9d5 ⬆️(frontend) bump to react 19.0.0
Last version of Blocknotes is compatible with
React 19.0.0, it seems even necessary to
bump the version of React to 19.0.0.
We bump the version of React to 19.0.0 and
remove the react packages from renovate
list of ignored dependencies.
2025-03-10 09:26:19 +01:00
Anthony LC
1c6d18fdf3 📌(frontend) pin yjs globally
We had a warning about yjs multiple versions
between dependencies. We pinned yjs globally
to avoid this warning and potential side effects.
2025-03-10 09:26:19 +01:00
Anthony LC
24d126f410 🚨(frontend) fix linter
- fix linter
- remove unnecessary style files
2025-03-10 09:26:19 +01:00
renovate[bot]
a5e1751cf3 ⬆️(dependencies) update js dependencies 2025-03-10 09:26:19 +01:00
Manuel Raynaud
0cabb655ad 🔒️(back) restrict access to favorite_list endpoint
favorite_list endpoint is accessible to anonymous user. This lead to an
error 500. This endpoint should be accessible only to authenticated
users.
2025-03-07 09:04:44 +01:00
Manuel Raynaud
38eb6d45b7 🐛(back) fix ValidationError exception handler
The exception handler for the ValidationError was not testing correctly
the existence of some attributes like `message_dict`.
2025-03-07 09:04:44 +01:00
Anthony LC
5bb7ad643a 🔖(minor) release 2.4.0
Added:
- (frontend) synchronize language-choice

Changed:
- Use sentry tags instead of extra scope

Fixed:
- 🐛(frontend) fix collaboration error
2025-03-06 15:59:34 +01:00
Anthony LC
57b8881fc6 📌(frontend) pin blocknote globally
Blocknote does not pinned the version.
We get bumped version instead of the version we want.
We pin the version of blocknote globally to
avoid this issue.
2025-03-06 15:09:17 +01:00
Anthony LC
89ad610ba6 🐛(frontend) fix collaboration error
We upgrade blocknote to 0.23.2-hotfix.0,
it includes a fix with the collaboration.
2025-03-06 15:09:17 +01:00
rvveber
251787b835 🚨(tests) add language related tests; fix getByRole not working in tests
- adds tests and test-utility for solid language switching in tests
- fixes where ...getByRole(menuitem... would not return a valid object
2025-03-05 14:29:24 +01:00
rvveber
f95173e096 🐛(frontend) allow left panel to update on language change
- fixes a bug where after language-sync the left panel would remain
  in the same language as before.
2025-03-05 14:29:24 +01:00
rvveber
a7944cce80 (app) get language from backend; set browser-detected language if null
- adds useLanguageSynchronizer hook to update the:
  1. frontend-language to the user-preference - if there is one.
  2. user-preference to the (browser-detected) frontend-language - otherwise.
2025-03-05 14:29:24 +01:00
rvveber
7941fc91d5 🐛(frontend) set toolbar-popup to current language
- ensure editor is translated to i18n.resolvedLanguage => en
  as i18n.language could hold more accurate locale => en-GB etc..
2025-03-05 14:29:24 +01:00
rvveber
7fc83a4fcd (frontend) the LanguagePicker now uses config as options
- config endpoint languages are used as available options for LanguagePicker
- updating the language from it, triggers an update on the user via API
2025-03-05 14:29:24 +01:00
rvveber
2bf47b7705 (backend) the LanguagePicker now uses config as options
- config endpoint languages are used as available options for LanguagePicker
- updating the language from it, triggers an update on the user via API
2025-03-05 14:29:24 +01:00
rvveber
23b0214a2a (frontend) add language utility for "locale"
- Adds a helper for working with locales
- More details in their annotations
- Unnecessary, if in the future, the backend uses
  the same locales as the keys in the translations (ISO 639-1)
2025-03-05 14:29:24 +01:00
rvveber
f244509de3 (frontend) add API access for 'language' attribute on User model
- allow the language attribute on the user to be updated via API
- add frontend function to update the user language via API
- extend defaults on the test users, to have fixed language in E2E tests
- extend types and variables using the types with the new field
2025-03-05 14:29:24 +01:00
rvveber
fda5f8f008 (backend) add API access for 'language' attribute on User model
- allow the language attribute on the user to be updated via API
- add frontend function to update the user language via API
- extend defaults on the test users, to have fixed language in E2E tests
- extend types and variables using the types with the new field
2025-03-05 14:29:24 +01:00
rvveber
9a79b09b07 🔨(backend) make the 'language' attribute on the User model nullable
- allow the language on the user to be unset
- set the default language to be unset
- helps us determine that the user has yet to set a language preference
2025-03-05 14:29:24 +01:00
rvveber
b24acd14e2 🔨(frontend) email invitation in invited user's language
- language for invitation emails => language saved on the invited user
- if invited user does not exist yet => language of the sending user
- if for some reason no sending user => system default language
2025-03-05 14:29:24 +01:00
rvveber
1531846115 🔨(backend) email invitation in invited user's language
- language for invitation emails => language saved on the invited user
- if invited user does not exist yet => language of the sending user
- if for some reason no sending user => system default language
2025-03-05 14:29:24 +01:00
Manuel Raynaud
ebf6d46e37 ♻️(front) use sentry tags instead of extra scope
To ease filtering issues on sentry, we want to use tags instead of extra
scope. Tags are indexed and searchable, it's not the case with extra
scope. Moreover using setEtra to add additional data is deprecated.
2025-03-05 10:26:23 +01:00
Manuel Raynaud
b9b5f86cf4 ️(back) restrict documents to restore using only the queryset
To determine the descendant to restore or not, we were looking building
a complex exclude clause. This can be simplify focusing only on data we
already have without making an extra query to fetch the list of
descendant to exclude.
2025-03-04 18:03:18 +01:00
renovate[bot]
56412b0be5 ⬆️(dependencies) update python dependencies 2025-03-04 14:24:58 +01:00
Anthony LC
af052cd06b 🔖(minor) release 2.3.0
Added:
- 💄(frontend) add error pages
- 🔒️ Manage unsafe attachments
- (frontend) Custom block quote with export
- (frontend) add open source section homepage

Changed:
- 🛂(frontend) Restore version visibility
- 📝(doc) minor README.md formatting and wording enhancements
- ♻️Stop setting a default title on doc creation
- ♻️(frontend) misc ui improvements

Fixed:
- 🐛(backend) allow any type of extensions for media download
- ♻️(frontend) improve table pdf rendering
2025-03-04 12:12:57 +01:00
Anthony LC
8927635c5f 💄(frontend) hide Crisp when mobile and modal
The Crisp button was hidding buttons on mobile
when a modal was open. This commit hides the
Crisp button when a modal is open on mobile.
2025-03-04 12:12:57 +01:00
Anthony LC
76bce4313b 🩹(frontend) fine tuning 2.3.0
- improve medium button style when 2 lines
- improve design on Firefox input title
- manage title modal without doc title
- improve redirect when 401
2025-03-04 12:12:57 +01:00
Anthony LC
5ac71bfac1 🐛(service-worker) update sw to create a doc without body
Offline creation of a doc was broken because we
don't add a default title anymore when we create a
doc, leading to POST requests without body.
we need to adapt the service worker to handle this
case.
2025-03-04 12:12:57 +01:00
Anthony LC
cb4e148afc ♻️(email) adapt email when no title
Default title is not set when we create a document
anymore. We need to adapt the email to handle
this case.
2025-03-04 12:12:57 +01:00
AntoLC
2d24825be0 🌐(i18n) update translated strings
Update translated files with new translations
2025-03-04 12:12:57 +01:00
Anthony LC
7b1ddc0e05 🛂(backend) remove svg from unsafe
We added content-security-policy on nginx.
It should be safe to allow svg files now.
We remove the svg file from the unsafe
attachments list. We adapt the tests accordingly.
2025-03-03 13:18:40 +01:00
Manuel Raynaud
22a665e535 🔒️(nginx) manage Content-Security-Policy in nginx config
The media route is managed by nginx. On this route we want to add the
Content-Security-Header to forbid fetching any resources.
See : https://content-security-policy.com/
2025-03-03 13:18:40 +01:00
Manuel Raynaud
a22bf95bce 🔒️(back) set ContentDisposition on media upload
On the media upload endpoint, we want to set the content-disposition
header. Its value is based on the uploaded file mime-type and if flagged
as unsafe. If the file is not an image or is unsafe then the
contentDisposition is set to attachment to force its download.
Otherwise, we set it to inline.
2025-03-03 13:18:40 +01:00
Anthony LC
3ce1826355 🚚(frontend) toolbar components in BlockNoteToolBar folder
We moved the toolbar components in BlockNoteToolBar
folder.
2025-03-03 13:18:40 +01:00
Anthony LC
d099d58f77 🛂(frontend) secure download button
Blocknote download button opens the file in a new
tab, which could be not secure because of XSS attacks.
We replace the download button with a new one that
downloads the file instead of opening it in a new tab.
Some files are flags as unsafe (SVG / js / exe),
for these files we add a confirmation modal before
downloading the file to prevent the user from
downloading a file that could be harmful.
In the future, we could add other security layers
from this model, to analyze the file before
downloading it by example.
2025-03-03 13:18:40 +01:00
Anthony LC
ebd49f05a8 🚸(frontend) block click on unsafe image
We want to prevent the user to open unsafe images
in the browser. We blocked the click on the images.
To download them, the user will have to use the
download button.
2025-03-03 13:18:40 +01:00
Anthony LC
315c2c2c43 🐛(frontend) improve authenticated state
It can happen that the user is authenticated
then the token is expired. The authenticated
state should be updated to false in this case.
2025-03-03 13:18:40 +01:00
Anthony LC
e442908c50 💄(frontend) improve the design of the alert error
Since the new design implementation,
the alert error was not looking good.
This commit improves the design of the alert error.
2025-03-03 13:18:40 +01:00
Anthony LC
6672292d93 🛂(backend) add unsafe in the attachments filename
The frontend cannot access custom headers of a file,
so we need to add a flag in the filename.
We add the `unsafe` flag in the filename to
indicate that the file is unsafe.

Previous filename: "/{UUID4}.{extension}"
New filename: "/{UUID4}-unsafe.{extension}"
2025-03-03 13:18:40 +01:00
Manuel Raynaud
7dda74421f ♻️(back) extract ancestor deleted_at directly from db in restore method
In the restore method, all the ancestors with a deleted_at date set are
extracted from the database and then the oldest value is extracted using
the min python function. This usage of min can be removed by sorting
directly the deleted_at at the databse level and then fetching the first
one. It's faster and easier to maintain.
2025-03-03 13:05:36 +01:00
Anthony LC
9c25b684e3 (frontend) add open source section homepage
We decided to add the open source section on
the homepage of Docs.
2025-03-03 12:42:18 +01:00
Anthony LC
cd5ee3fb7c (frontend) adapt export to quote block
We have a new block type, the quote block.
We have to adapt the export to handle this
new block type.
2025-03-03 12:27:02 +01:00
Anthony LC
942c0f059c 🏗️(frontend) blockMapping refactoring
As made for TablePDF, we separate the block mapping
in separate files. This will allow us to have
a better separation of concerns and to have
a more maintainable codebase.
We improve as well the typing. It will be easier
to add new blocks in the future.
2025-03-03 12:27:02 +01:00
Anthony LC
3acee1e6fa (frontend) create feature doc-export
Create the feature doc-export, it will be
responsible for exporting the document.
2025-03-03 12:27:02 +01:00
Anthony LC
26ea32bd0b 🏷️(frontend) adapt title types
We recently changed the default title behavior.
It can now be undefined, we have to change the
types accordingly.
2025-03-03 12:27:02 +01:00
Anthony LC
7f6ffa0123 (frontend) add quote blocks to the editor
Add a custom block to quote in the editor.
2025-03-03 12:27:02 +01:00
Samuel Paccoud - DINUM
ef2127585c 🐛(backend) allow any type of extensions for media download
The regex to validate media file extensions was too restrictive.
2025-03-03 11:21:41 +01:00
Anthony LC
54a75bc338 ♻️(frontend) update some elements
- add panel information when document is
authenticated
- add a copy link button in the toolbox
on the document
- fix when long title document
- modals fit design
- mobile responsive changes
2025-02-24 10:49:20 +01:00
Anthony LC
50d098c777 💄(frontend) adapt language picker design
Adapt the language picker design to
to fit with the new design of the app.
2025-02-24 10:49:20 +01:00
Anthony LC
757c09b189 ♻️(frontend) adapt other error pages to new design
Adapt the other error pages to the new design.
2025-02-24 10:27:42 +01:00
Anthony LC
30c5cfab62 💄(frontend) add page 401
Add a 401 page when user try to access a
doc that need authentication.
2025-02-24 10:27:42 +01:00
Anthony LC
f069329e18 💄(frontend) add page 403
Add a page and integrate the design for the
403 error page.
2025-02-24 10:27:42 +01:00
Manuel Raynaud
ef8ee67553 ♻️(compose) set compose name in the docker-compose file
The helper bin/compose was using the option -p to set the compose
project name but this option is not used in the Makefile. This can lead
to different way to use the docker compose file definition with
different project name. In order to have a consistent name everywhere
and for everybody, we set the name in the docker compose file itself.
2025-02-20 10:32:37 +01:00
Anthony LC
ad47fc2d60 (e2e) global setup authentication
Because of the parallelism of the tests,
the authentication setup was flaky. Sometimes
the tests would run before the authentication
was complete.
We change to a global setup instead of the
project dependency setup, it should be more
reliable.
We improved the waiting states of the authentication
setup.
2025-02-19 14:57:15 +01:00
Nathan Vasse
009f5d6ed4 ♻️(front) improve table pdf rendering
The previous way of rendering table was causing issues when tables
could not fit on one page. I then came accross this discussion
https://github.com/diegomura/react-pdf/issues/2343. The author
created a lib to improve the rendering of table, it's better, but
still not perfect maybe.
2025-02-19 13:58:29 +01:00
Nathan Vasse
64d0072c8d 🐛(front) improve text rendering in pdf
The rendered text had unwanted line breaks in middle of them.
It was because we were not using the appropriate Text component, the
one to be used in the one from react-pdf.
2025-02-18 17:01:58 +01:00
Anthony LC
aefbc2e0b9 🛂(frontend) hide version restore button when reader
When you are a reader member, you have the right to
see the version but you cannot restore
them. So, we hide the restore button
when you are a reader.
2025-02-18 09:30:37 +01:00
Anthony LC
15dc1e3012 🦺(migration) add back the migration folders to linter
Previous commit add "core/tests/migrations".
The linter could not pass on it because all the
migration folders were excluded from the linter.
We remove this exclusion, tests and migrations can
now be linted and formatted automatically.
2025-02-15 23:39:17 +01:00
Anthony LC
6cc20aeacb 🩹(migration) add migration to update default titles
The frontend was setting a default titles for
documents with empty titles.
This migration updates the document table to set
the title to null instead of the default title.
We add a test to ensure that the migration
works as expected.
2025-02-15 23:39:17 +01:00
Anthony LC
7da7214afb (backend) add django-test-migrations
Add django-test-migrations to the project.
It is a tool that helps to test Django migrations.
2025-02-15 23:39:17 +01:00
Anthony LC
c369419512 ♻️(frontend) stop setting a default title on doc creation
We were setting a default title to our document
during creation, but we should not do that,
it created lot of similar titles, lot of
documents will show up during search.
2025-02-15 23:39:17 +01:00
Anthony LC
d9ad397c94 🩹(frontend) minor fixes
- fix linter warning on one e2e test
- improve logo svg
- improve cursor
- improve grid loader
2025-02-15 23:39:17 +01:00
Manuel Raynaud
3191d890f3 ♻️(docker) rename frontend-dev service in frontend
The frontend-dev service is in fact using the production image. We
rename it in frontend accordingly with what it really does. We also have
to change name rules in Makefile to be consistent.
2025-02-14 19:54:38 +01:00
Manuel Raynaud
68f3387539 ♻️(make) make run command starting everything
The run command is not starting the frontend application. We change the
run commands. The run command is strating everything. The run-backend
command is starting all services needed to use the backend application.
2025-02-14 19:54:38 +01:00
Manuel Raynaud
0dc8b4556c ♻️(docker) remove usage of dockerize
We remove dockerize and use healthcheck on docker compose services
instead.
2025-02-14 19:54:38 +01:00
Manuel Raynaud
e123e91959 🐛(nginx) increase nginx buffer size when proxifying keycloak
Nginx is used to proxify keycloak in our development configuration. When
a new user is created keycloak is send a large amount of headers in its
response and the default nginx config is not enough to handle this
amount of headers. We have to increase the proxy buffer size to handle
them.
2025-02-14 19:54:38 +01:00
Bastien Guerry
2709400773 🔊(changelog) add a changelog entry
Add "📝(doc) minor README.md formatting and wording enhancements" in
the unreleased section.
2025-02-14 17:39:52 +01:00
Bastien Guerry
8281c6159b 📝(doc) minor README.md formatting and wording enhancements
See https://github.com/suitenumerique/docs/issues/622
2025-02-14 17:39:52 +01:00
Anthony LC
296dbb7957 🔖(minor) release 2.2.0
Added:
- 📝(doc) Add security.md and codeofconduct.md
- (frontend) add home page
- (frontend) cursor display on activity
- (frontend) Add export page break

Changed:
- 🔧(backend) make AI feature reach configurable

Fixed:
- 🌐(CI) Fix email partially translated
- 🐛(frontend) fix cursor breakline
- 🐛(frontend) fix style pdf export
2025-02-11 14:16:58 +01:00
Anthony LC
3827f0f799 🩹(frontend) fine tuning v2.2.0
Fix minor issues:
- Add some height to the carret in the editor
- Improve css in mail templates
- Improve images resolution on homepage
2025-02-11 14:16:58 +01:00
Anthony LC
d89e3dc6d4 🛂(frontend) display the AI buttons depend abilities
Anybody with edit right could use the AI.
We changed this behavior, now we have to be
authentified with edit right.
We update the UI to display the AI buttons
only if the user has the correct AI ability.
2025-02-11 10:54:55 +01:00
Samuel Paccoud - DINUM
91cf5f9367 🔧(backend) make AI feature reach configurable
We want to be able to define whether AI features are available to
anonymous users who gained editor access on a document, or if we
demand that they be authenticated or even if we demand that they
gained their editor access via a specific document access.

Being authenticated is now the default value. This will change the
default behavior on your existing instance (see UPGRADE.md)
2025-02-11 10:54:55 +01:00
Anthony LC
5cc4b07cf6 💄(frontend) improve caret style
Improve the caret, to looks more like the
google doc caret.
2025-02-10 15:53:03 +01:00
Anthony LC
0cfc242e09 🚚(frontend) move Blocknote styles
Move Blocknote styles to a separate file.
2025-02-10 15:53:03 +01:00
Anthony LC
a6b3cfdb0c (frontend) add export page break
Blocknotejs introduced the ability to export a
document with page breaks.
This commit adds the page break feature to the
editor and so to our export feature.
2025-02-10 15:53:03 +01:00
Anthony LC
5ead18c94c 💬(frontend) add lacking buttons open source section
Some buttons were lacking in the open source
section of the home page. This commit adds them.
2025-02-10 15:40:23 +01:00
AntoLC
5eeb8cae5c 🌐(i18n) update translated strings
Update translated files with new translations
2025-02-10 15:40:23 +01:00
Jacques ROUSSEL
68bf024005 (helm) add pdbs to deployments
In order to avoid a service interruption during a Kubernetes (k8s)
upgrade, we add a Pod Disruption Budget (PDB) to deployments.
2025-02-10 13:05:54 +01:00
Anthony LC
fdd1068c90 (frontend) fix test copy html
The recent upgrade of Blocknote has caused the
test to fail.
This commit updates the test to reflect
the new html structure.
2025-02-10 10:50:29 +01:00
Anthony LC
ba695bf647 ⬇️(frontend) downgrade to @react-pdf/renderer to 4.1.6
The version 4.2.1 of @react-pdf/renderer
is not compatible @blocknote/xl-pdf-exporter
and @blocknote/xl-docx-exporter.
2025-02-10 10:50:29 +01:00
Anthony LC
27e7aec193 💄(frontend) improve styles export pdf
When exporting a document to PDF, the headings
spacings were too small, the break lines were
not displayed. This commit fixes these issues
by replacing the needed blocks.
2025-02-10 10:50:29 +01:00
Anthony LC
58b712a1de 🐛(frontend) fix cursor breakline
We had breakline issues with the initial
cursor because of some css properties.
We changed the cursor css to not take
any space in the lines,
avoiding the breakline issues.
We keep the new cursor visibility
feature (always, activity).
2025-02-10 10:50:29 +01:00
renovate[bot]
08f9036523 ⬆️(dependencies) update js dependencies 2025-02-10 10:50:29 +01:00
Anthony LC
ebe3efc8f7 💄(frontend) update the favicon
Update the favicon with a better one.
2025-02-07 18:18:22 +01:00
Anthony LC
66fbf27913 🩹(frontend) fix PageLayout
The page layout was rendered behind the header,
which caused the top of the mention legales pages
to be hidden.
This commit fixes this issue.
2025-02-07 18:18:22 +01:00
Anthony LC
20e4a4e42a 🥚(frontend) easter egg in dev tools
Add a easter egg in the browser dev tools.
2025-02-07 18:18:22 +01:00
Anthony LC
1aa4844eeb 🎨(frontend) add dsfr proconnect homepage
If we are with the DSFR theme, we need to add the
proconnect button to the homepage.
We add an option in the cunningham theme to
display the proconnect section instead of the
opensource section.
2025-02-07 18:18:22 +01:00
Nathan Panchout
4bb9c092cb (frontend) add white label homepage
Add white label homepage with new assets and
components. When the user is not logged in,
the homepage will be displayed.
2025-02-07 18:18:22 +01:00
Anthony LC
c493eb8924 ♻️(frontend) use a hook instead of a store for auth
We will use a hook instead of a store for the auth
feature. The hook will be powered by ReactQuery,
it will provide us fine-grained control over the
auth state and will be easier to use.
2025-02-07 18:18:22 +01:00
Anthony LC
40fdf97520 🚚(frontend) move auth to its own feature
We will move auth to its own feature to make it
easier to manage and to make it more modular.
2025-02-07 18:18:22 +01:00
Anthony LC
91b10e75dd 💄(email) fix line height email title
The line height of the email title was not
the correct size. We let the title managing
its own line height.
2025-02-07 17:05:49 +01:00
Anthony LC
7a6da10e1c 🐛(i18n) add back the missing email translations
We changed the way we upload the translations to
Crowdin, some translations were missing for the
email templates. We add them back and improve
the tests to make sure we don't forget them again.
2025-02-07 17:05:49 +01:00
Anthony LC
004e8ec645 🌐(CI) build mails when crowdin_upload workflow
When we were executing the crowdin_upload workflow,
we were not building the mail template to dispatch it
to the backend. It resulted in the mail not being
totally translated. This commit fixes that issue
by adding the build mail step to the crowdin_upload.
To do so, we added it to the dependencies workflow.
"dependencies" workflow is callable by other
workflows that need a specific job.
2025-02-07 17:05:49 +01:00
607 changed files with 44414 additions and 16808 deletions

View File

@@ -34,3 +34,4 @@ db.sqlite3
# Frontend
node_modules
.next

23
.gitattributes vendored Normal file
View File

@@ -0,0 +1,23 @@
# Set the default behavior for all files
* text=auto eol=lf
# Binary files (should not be modified)
*.png binary
*.jpg binary
*.jpeg binary
*.gif binary
*.ico binary
*.mov binary
*.mp4 binary
*.mp3 binary
*.flv binary
*.fla binary
*.swf binary
*.gz binary
*.zip binary
*.7z binary
*.ttf binary
*.woff binary
*.woff2 binary
*.eot binary
*.pdf binary

View File

@@ -1,7 +1,7 @@
---
name: 🐛 Bug Report
about: If something is not working as expected 🤔.
labels: ["bug", "triage"]
---
## Bug Report
@@ -18,8 +18,8 @@ A clear and concise description of what you expected to happen (or code).
3. And then the bug happens!
**Environment**
- Impress version:
- Platform:
- Docs version:
- Instance url:
**Possible Solution**
<!--- Only if you have suggestions on a fix for the bug -->

View File

@@ -1,7 +1,7 @@
---
name: ✨ Feature Request
about: I have a suggestion (and may want to build it 💪)!
labels: ["feature", "triage"]
---
## Feature Request
@@ -16,8 +16,8 @@ A clear and concise description of what you want to happen. Add any considered d
A clear and concise description of any alternative solutions or features you've considered.
**Discovery, Documentation, Adoption, Migration Strategy**
If you can, explain how users will be able to use this and possibly write out a version the docs (if applicable).
Maybe a screenshot or design?
If you can, explain how users will be able to use this and possibly write out some documentation (if applicable).
Maybe add a screenshot or design?
**Do you want to work on it through a Pull Request?**
<!-- Make sure to coordinate with us before you spend too much time working on an implementation! -->

View File

@@ -1,17 +1,13 @@
---
name: 🤗 Support Question
about: If you have a question 💬, or something was not clear from the docs!
labels: ["support", "triage"]
---
## Support request
**Checks before filing**
Please make sure you have read our [main Readme](https://github.com/suitenumerique/docs).
<!-- ^ Click "Preview" for a nicer view! ^
We primarily use GitHub as an issue tracker. If however you're encountering an issue not covered in the docs, we may be able to help! -->
---
Please make sure you have read our [main Readme](https://github.com/numerique-gouv/impress).
Also make sure it was not already answered in [an open or close issue](https://github.com/numerique-gouv/impress/issues).
Also make sure it was not already answered in [an open or close issue](https://github.com/suitenumerique/docs/issues?q=is%3Aissue%20state%3Aopen%20label%3Asupport).
If your question was not covered, and you feel like it should be, fire away! We'd love to improve our docs! 👌

View File

@@ -1,11 +1,22 @@
## Purpose
Description...
Describe the purpose of this pull request.
## Proposal
Description...
- [ ] item 1...
- [ ] item 2...
- [] item 1...
- [] item 2...
## External contributions
Thank you for your contribution! 🎉
Please ensure the following items are checked before submitting your pull request:
- [ ] I have read and followed the [contributing guidelines](https://github.com/suitenumerique/docs/blob/main/CONTRIBUTING.md)
- [ ] I have read and agreed to the [Code of Conduct](https://github.com/suitenumerique/docs/blob/main/CODE_OF_CONDUCT.md)
- [ ] I have signed off my commits with `git commit --signoff` (DCO compliance)
- [ ] I have signed my commits with my SSH or GPG key (`git commit -S`)
- [ ] My commit messages follow the required format: `<gitmoji>(type) title description`
- [ ] I have added a changelog entry under `## [Unreleased]` section (if noticeable change)
- [ ] I have added corresponding tests for new features or bug fixes (if applicable)

View File

@@ -7,10 +7,11 @@ on:
- 'release/**'
jobs:
install-front:
uses: ./.github/workflows/front-dependencies-installation.yml
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
node_version: '20.x'
node_version: '22.x'
with-front-dependencies-installation: true
synchronize-with-crowdin:
runs-on: ubuntu-latest

View File

@@ -7,13 +7,15 @@ on:
- main
jobs:
install-front:
uses: ./.github/workflows/front-dependencies-installation.yml
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
node_version: '20.x'
node_version: '22.x'
with-front-dependencies-installation: true
with-build_mails: true
synchronize-with-crowdin:
needs: install-front
needs: install-dependencies
runs-on: ubuntu-latest
steps:
@@ -23,12 +25,19 @@ jobs:
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: "3.12.6"
python-version: "3.13.3"
- name: Upgrade pip and setuptools
run: pip install --upgrade pip setuptools
- name: Install development dependencies
run: pip install --user .
working-directory: src/backend
- name: Restore the mail templates
uses: actions/cache@v4
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
fail-on-cache-miss: true
- name: Install gettext
run: |
sudo apt-get update

85
.github/workflows/dependencies.yml vendored Normal file
View File

@@ -0,0 +1,85 @@
name: Dependency reusable workflow
on:
workflow_call:
inputs:
node_version:
required: false
default: '22.x'
type: string
with-front-dependencies-installation:
type: boolean
default: false
with-build_mails:
type: boolean
default: false
jobs:
front-dependencies-installation:
if: ${{ inputs.with-front-dependencies-installation == true }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Setup Node.js
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node_version }}
- name: Install dependencies
if: steps.front-node_modules.outputs.cache-hit != 'true'
run: cd src/frontend/ && yarn install --frozen-lockfile
- name: Cache install frontend
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
build-mails:
if: ${{ inputs.with-build_mails == true }}
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/mail
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore the mail templates
uses: actions/cache@v4
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
- name: Setup Node.js
if: steps.mail-templates.outputs.cache-hit != 'true'
uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node_version }}
- name: Install yarn
if: steps.mail-templates.outputs.cache-hit != 'true'
run: npm install -g yarn
- name: Install node dependencies
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn install --frozen-lockfile
- name: Build mails
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn build
- name: Cache mail templates
if: steps.mail-templates.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}

View File

@@ -6,11 +6,13 @@ on:
push:
branches:
- 'main'
- 'feature/doc-dnd'
tags:
- 'v*'
pull_request:
branches:
- 'main'
- 'ci/trivy-fails'
env:
DOCKER_USER: 1001:127
@@ -38,7 +40,6 @@ jobs:
with:
docker-build-args: '--target backend-production -f Dockerfile'
docker-image-name: 'docker.io/lasuite/impress-backend:${{ github.sha }}'
continue-on-error: true
-
name: Build and push
uses: docker/build-push-action@v6
@@ -72,7 +73,6 @@ jobs:
with:
docker-build-args: '-f src/frontend/Dockerfile --target frontend-production'
docker-image-name: 'docker.io/lasuite/impress-frontend:${{ github.sha }}'
continue-on-error: true
-
name: Build and push
uses: docker/build-push-action@v6
@@ -80,7 +80,9 @@ jobs:
context: .
file: ./src/frontend/Dockerfile
target: frontend-production
build-args: DOCKER_USER=${{ env.DOCKER_USER }}:-1000
build-args: |
DOCKER_USER=${{ env.DOCKER_USER }}:-1000
PUBLISH_AS_MIT=false
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
@@ -106,8 +108,7 @@ jobs:
uses: numerique-gouv/action-trivy-cache@main
with:
docker-build-args: '-f src/frontend/servers/y-provider/Dockerfile --target y-provider'
docker-image-name: 'docker.io/lasuite/impress-frontend:${{ github.sha }}'
continue-on-error: true
docker-image-name: 'docker.io/lasuite/impress-y-provider:${{ github.sha }}'
-
name: Build and push
uses: docker/build-push-action@v6
@@ -125,15 +126,11 @@ jobs:
- build-and-push-frontend
- build-and-push-backend
runs-on: ubuntu-latest
if: |
github.event_name != 'pull_request'
if: github.event_name != 'pull_request'
steps:
-
name: Checkout repository
uses: actions/checkout@v4
-
name: Call argocd github webhook
run: |
data='{"ref": "'$GITHUB_REF'","repository": {"html_url":"'$GITHUB_SERVER_URL'/'$GITHUB_REPOSITORY'"}}'
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac ''${{ secrets.ARGOCD_PREPROD_WEBHOOK_SECRET}}'' | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" ${{ vars.ARGOCD_PREPROD_WEBHOOK_URL }}
- uses: numerique-gouv/action-argocd-webhook-notification@main
id: notify
with:
deployment_repo_path: "${{ secrets.DEPLOYMENT_REPO_URL }}"
argocd_webhook_secret: "${{ secrets.ARGOCD_PREPROD_WEBHOOK_SECRET }}"
argocd_url: "${{ vars.ARGOCD_PREPROD_WEBHOOK_URL }}"

View File

@@ -1,36 +0,0 @@
name: Install frontend installation reusable workflow
on:
workflow_call:
inputs:
node_version:
required: false
default: '20.x'
type: string
jobs:
front-dependencies-installation:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Setup Node.js
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node_version }}
- name: Install dependencies
if: steps.front-node_modules.outputs.cache-hit != 'true'
run: cd src/frontend/ && yarn install --frozen-lockfile
- name: Cache install frontend
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}

View File

@@ -11,7 +11,7 @@ jobs:
helmfile-lint:
runs-on: ubuntu-latest
container:
image: ghcr.io/helmfile/helmfile:latest
image: ghcr.io/helmfile/helmfile:v0.171.0
steps:
-
name: Checkout repository
@@ -22,9 +22,9 @@ jobs:
run: |
set -e
HELMFILE=src/helm/helmfile.yaml
environments=$(awk '/environments:/ {flag=1; next} flag && NF {print} !NF {flag=0}' "$HELMFILE" | grep -E '^[[:space:]]{2}[a-zA-Z]+' | sed 's/^[[:space:]]*//;s/:.*//')
environments=$(awk 'BEGIN {in_env=0} /^environments:/ {in_env=1; next} /^---/ {in_env=0} in_env && /^ [^ ]/ {gsub(/^ /,""); gsub(/:.*$/,""); print}' "$HELMFILE")
for env in $environments; do
echo "################### $env lint ###################"
helmfile -e $env -f $HELMFILE lint || exit 1
echo -e "\n"
done
done

View File

@@ -10,13 +10,14 @@ on:
jobs:
install-front:
uses: ./.github/workflows/front-dependencies-installation.yml
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
node_version: '20.x'
node_version: '22.x'
with-front-dependencies-installation: true
test-front:
needs: install-front
needs: install-dependencies
runs-on: ubuntu-latest
steps:
- name: Checkout repository
@@ -25,7 +26,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20.x"
node-version: "22.x"
- name: Restore the frontend cache
uses: actions/cache@v4
@@ -39,7 +40,7 @@ jobs:
lint-front:
runs-on: ubuntu-latest
needs: install-front
needs: install-dependencies
steps:
- name: Checkout repository
uses: actions/checkout@v4
@@ -47,7 +48,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20.x"
node-version: "22.x"
- name: Restore the frontend cache
uses: actions/cache@v4
with:
@@ -60,7 +61,7 @@ jobs:
test-e2e-chromium:
runs-on: ubuntu-latest
needs: install-front
needs: install-dependencies
timeout-minutes: 20
steps:
- name: Checkout repository
@@ -69,7 +70,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20.x"
node-version: "22.x"
- name: Restore the frontend cache
uses: actions/cache@v4
@@ -85,29 +86,7 @@ jobs:
run: cd src/frontend/apps/e2e && yarn install --frozen-lockfile && yarn install-playwright chromium
- name: Start Docker services
run: make bootstrap FLUSH_ARGS='--no-input' cache=
# Tool to wait for a service to be ready
- name: Install Dockerize
run: |
curl -sSL https://github.com/jwilder/dockerize/releases/download/v0.8.0/dockerize-linux-amd64-v0.8.0.tar.gz | sudo tar -C /usr/local/bin -xzv
- name: Wait for services to be ready
run: |
printf "Minio check...\n"
dockerize -wait tcp://localhost:9000 -timeout 20s
printf "Keyclock check...\n"
dockerize -wait tcp://localhost:8080 -timeout 20s
printf "Server collaboration check...\n"
dockerize -wait tcp://localhost:4444 -timeout 20s
printf "Ngnix check...\n"
dockerize -wait tcp://localhost:8083 -timeout 20s
printf "DRF check...\n"
dockerize -wait tcp://localhost:8071 -timeout 20s
printf "Postgres Keyclock check...\n"
dockerize -wait tcp://localhost:5433 -timeout 20s
printf "Postgres back check...\n"
dockerize -wait tcp://localhost:15432 -timeout 20s
run: make bootstrap-e2e FLUSH_ARGS='--no-input'
- name: Run e2e tests
run: cd src/frontend/ && yarn e2e:test --project='chromium'
@@ -130,7 +109,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20.x"
node-version: "22.x"
- name: Restore the frontend cache
uses: actions/cache@v4
@@ -146,7 +125,7 @@ jobs:
run: cd src/frontend/apps/e2e && yarn install --frozen-lockfile && yarn install-playwright firefox webkit chromium
- name: Start Docker services
run: make bootstrap FLUSH_ARGS='--no-input' cache=
run: make bootstrap-e2e FLUSH_ARGS='--no-input'
- name: Run e2e tests
run: cd src/frontend/ && yarn e2e:test --project=firefox --project=webkit

View File

@@ -9,6 +9,11 @@ on:
- "*"
jobs:
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
with-build_mails: true
lint-git:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request' # Makes sense only for pull requests
@@ -56,45 +61,24 @@ jobs:
exit 1
fi
build-mails:
lint-spell-mistakes:
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/mail
if: github.event_name == 'pull_request'
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: "18"
- name: Restore the mail templates
uses: actions/cache@v4
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
- name: Install yarn
if: steps.mail-templates.outputs.cache-hit != 'true'
run: npm install -g yarn
- name: Install node dependencies
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn install --frozen-lockfile
- name: Build mails
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn build
- name: Cache mail templates
if: steps.mail-templates.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
uses: actions/checkout@v2
- name: Install codespell
run: pip install --user codespell
- name: Check for typos
run: |
codespell \
--check-filenames \
--ignore-words-list "Dokument,afterAll,excpt,statics" \
--skip "./git/" \
--skip "**/*.po" \
--skip "**/*.pot" \
--skip "**/*.json" \
--skip "**/yarn.lock"
lint-back:
runs-on: ubuntu-latest
@@ -107,7 +91,7 @@ jobs:
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: "3.12.6"
python-version: "3.13.3"
- name: Upgrade pip and setuptools
run: pip install --upgrade pip setuptools
- name: Install development dependencies
@@ -121,7 +105,7 @@ jobs:
test-back:
runs-on: ubuntu-latest
needs: build-mails
needs: install-dependencies
defaults:
run:
@@ -169,6 +153,7 @@ jobs:
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
fail-on-cache-miss: true
- name: Start MinIO
run: |
@@ -201,7 +186,7 @@ jobs:
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: "3.12.6"
python-version: "3.13.3"
- name: Install development dependencies
run: pip install --user .[dev]

View File

@@ -6,16 +6,272 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0),
and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Added
- ✨(frontend) add customization for translations #857
- 📝(project) add troubleshoot doc #1066
- 📝(project) add system-requirement doc #1066
- 🔧(front) configure x-frame-options to DENY in nginx conf #1084
- (doc) add documentation to install with compose #855
- ✨(backend) allow to disable checking unsafe mimetype on attachment upload
- ✨Ask for access #1081
### Changed
- 📌(yjs) stop pinning node to minor version on yjs docker image #1005
- 🧑‍💻(docker) add .next to .dockerignore #1055
- 🧑‍💻(docker) handle frontend development images with docker compose #1033
- 🧑‍💻(docker) add y-provider config to development environment #1057
- ⚡️(frontend) optimize document fetch error handling #1089
### Fixed
- 🐛(frontend) table of content disappearing #982
- 🐛(frontend) fix multiple EmojiPicker #1012
- 🐛(frontend) fix meta title #1017
- 🔧(git) set LF line endings for all text files #1032
- 📝(docs) minor fixes to docs/env.md
- ✨(backend) support `_FILE` environment variables for secrets #912
- ✨(frontend) support `_FILE` environment variables for secrets #912
### Removed
- 🔥(frontend) remove Beta from logo #1095
## [3.3.0] - 2025-05-06
### Added
- ✨(backend) add endpoint checking media status #984
- ✨(backend) allow setting session cookie age via env var #977
- ✨(backend) allow theme customnization using a configuration file #948
- ✨(frontend) Add a custom callout block to the editor #892
- 🚩(frontend) version MIT only #911
- ✨(backend) integrate maleware_detection from django-lasuite #936
- 🏗️(frontend) Footer configurable #959
- 🩺(CI) add lint spell mistakes #954
- ✨(frontend) create generic theme #792
- 🛂(frontend) block edition to not connected users #945
- 🚸(frontend) Let loader during upload analyze #984
- 🚩(frontend) feature flag on blocking edition #997
### Changed
- 📝(frontend) Update documentation #949
- ✅(frontend) Improve tests coverage #949
- ⬆️(docker) upgrade backend image to python 3.13 #973
- ⬆️(docker) upgrade node images to alpine 3.21 #973
### Fixed
- 🐛(y-provider) increase JSON size limits for transcription conversion #989
### Removed
- 🔥(back) remove footer endpoint #948
## [3.2.1] - 2025-05-06
## Fixed
- 🐛(frontend) fix list copy paste #943
- 📝(doc) update contributing policy (commit signatures are now mandatory) #895
## [3.2.0] - 2025-05-05
## Added
- ✨(frontend) multi-pages #701
- ✨(backend) include ancestors accesses on document accesses list view #846
- ✨(backend) add ancestors links reach and role to document API #846
- 🚸(backend) make document search on title accent-insensitive #874
- 🚩 add homepage feature flag #861
- 📝(doc) update contributing policy (commit signatures are now mandatory) #895
- ✨(settings) Allow configuring PKCE for the SSO #886
- 🌐(i18n) activate chinese and spanish languages #884
- 🔧(backend) allow overwriting the data directory #893
- (backend) add `django-lasuite` dependency #839
- ✨(frontend) advanced table features #908
## Changed
- ♻️(backend) stop requiring owner for non-root documents #846
- ♻️(backend) simplify roles by ranking them and return only the max role #846
- ⚡️(frontend) reduce unblocking time for config #867
- ♻️(frontend) bind UI with ability access #900
- ♻️(frontend) use built-in Quote block #908
## Fixed
- 🐛(backend) fix link definition select options linked to ancestors #846
- 🐛(nginx) fix 404 when accessing a doc #866
- 🔒️(drf) disable browsable HTML API renderer #919
- 🔒(frontend) enhance file download security #889
- 🐛(backend) race condition create doc #633
- 🐛(frontend) fix breaklines in custom blocks #908
## Fixed
- 🐛(backend) fix link definition select options linked to ancestors #846
## [3.1.0] - 2025-04-07
## Added
- ✨(backend) add ancestors links definitions to document abilities #846
- 🚩(backend) add feature flag for the footer #841
- 🔧(backend) add view to manage footer json #841
- ✨(frontend) add custom css style #771
- 🚩(frontend) conditionally render AI button only when feature is enabled #814
## Changed
- 🚨(frontend) block button when creating doc #749
## Fixed
- 🐛(backend) fix link definition select options linked to ancestors #846
- 🐛(back) validate document content in serializer #822
- 🐛(frontend) fix selection click past end of content #840
## [3.0.0] - 2025-03-28
## Added
- 📄(legal) Require contributors to sign a DCO #779
## Changed
- ♻️(frontend) Integrate UI kit #783
- 🏗️(y-provider) manage auth in y-provider app #804
## Fixed
- 🐛(backend) compute ancestor_links in get_abilities if needed #725
- 🔒️(back) restrict access to document accesses #801
## [2.6.0] - 2025-03-21
## Added
- 📝(doc) add publiccode.yml #770
## Changed
- 🚸(frontend) ctrl+k modal not when editor is focused #712
## Fixed
- 🐛(back) allow only images to be used with the cors-proxy #781
- 🐛(backend) stop returning inactive users on the list endpoint #636
- 🔒️(backend) require at least 5 characters to search for users #636
- 🔒️(back) throttle user list endpoint #636
- 🔒️(back) remove pagination and limit to 5 for user list endpoint #636
## [2.5.0] - 2025-03-18
## Added
- 📝(doc) Added GNU Make link to README #750
- ✨(frontend) add pinning on doc detail #711
- 🚩(frontend) feature flag analytic on copy as html #649
- ✨(frontend) Custom block divider with export #698
- 🌐(i18n) activate dutch language #742
- ✨(frontend) add Beautify action to AI transform #478
- ✨(frontend) add Emojify action to AI transform #478
## Changed
- 🧑‍💻(frontend) change literal section open source #702
- ♻️(frontend) replace cors proxy for export #695
- 🚨(gitlint) Allow uppercase in commit messages #756
- ♻️(frontend) Improve AI translations #478
## Fixed
- 🐛(frontend) SVG export #706
- 🐛(frontend) remove scroll listener table content #688
- 🔒️(back) restrict access to favorite_list endpoint #690
- 🐛(backend) refactor to fix filtering on children
and descendants views #695
- 🐛(action) fix notify-argocd workflow #713
- 🚨(helm) fix helmfile lint #736
- 🚚(frontend) redirect to 401 page when 401 error #759
## [2.4.0] - 2025-03-06
## Added
- ✨(frontend) synchronize language-choice #401
## Changed
- Use sentry tags instead of extra scope
## Fixed
- 🐛(frontend) fix collaboration error #684
## [2.3.0] - 2025-03-03
## Added
- ✨(backend) limit link reach/role select options depending on ancestors #645
- ✨(backend) add new "descendants" action to document API endpoint #645
- ✨(backend) new "tree" action on document detail endpoint #645
- ✨(backend) allow forcing page size within limits #645
- 💄(frontend) add error pages #643
- 🔒️ Manage unsafe attachments #663
- ✨(frontend) Custom block quote with export #646
- ✨(frontend) add open source section homepage #666
- ✨(frontend) synchronize language-choice #401
## Changed
- 🛂(frontend) Restore version visibility #629
- 📝(doc) minor README.md formatting and wording enhancements
-Stop setting a default title on doc creation #634
- ♻️(frontend) misc ui improvements #644
## Fixed
- 🐛(backend) allow any type of extensions for media download #671
- ♻️(frontend) improve table pdf rendering
- 🐛(email) invitation emails in receivers language
## [2.2.0] - 2025-02-10
## Added
- 📝(doc) Add security.md and codeofconduct.md #604
- ✨(frontend) add home page #608
- ✨(frontend) cursor display on activity #609
- ✨(frontend) Add export page break #623
## Changed
- 🔧(backend) make AI feature reach configurable #628
## Fixed
- 🌐(CI) Fix email partially translated #616
- 🐛(frontend) fix cursor breakline #609
- 🐛(frontend) fix style pdf export #609
## [2.1.0] - 2025-01-29
## Added
- ✨(backend) add duplicate action to the document API endpoint
- ⚗️(backend) add util to extract text from base64 yjs document
- ✨(backend) add soft delete and restore API endpoints to documents #516
- ✨(backend) allow organizing documents in a tree structure #516
- ✨(backend) add "excerpt" field to document list serializer #516
@@ -27,7 +283,7 @@ and this project adheres to
## Changed
- 💄(frontend) add abilities on doc row #581
- 💄(frontend) improve DocsGridItem responsive padding #582
- 💄(frontend) improve DocsGridItem responsive padding #582
- 🔧(backend) Bump maximum page size to 200 #516
- 📝(doc) Improve Read me #558
@@ -39,7 +295,6 @@ and this project adheres to
- 🔥(backend) remove "content" field from list serializer # 516
## [2.0.1] - 2025-01-17
## Fixed
@@ -94,12 +349,11 @@ and this project adheres to
- ⚡️(e2e) reduce flakiness on e2e tests #511
## Fixed
- 🐛(frontend) update doc editor height #481
- 💄(frontend) add doc search #485
## [1.9.0] - 2024-12-11
## Added
@@ -121,21 +375,18 @@ and this project adheres to
- 🐛(frontend) Fix hidden menu on Firefox #468
- 🐛(backend) fix sanitize problem IA #490
## [1.8.2] - 2024-11-28
## Changed
- ♻️(SW) change strategy html caching #460
## [1.8.1] - 2024-11-27
## Fixed
- 🐛(frontend) link not clickable and flickering firefox #457
## [1.8.0] - 2024-11-25
## Added
@@ -164,7 +415,6 @@ and this project adheres to
- 🐛(frontend) users have view access when revoked #387
- 🐛(frontend) fix placeholder editable when double clicks #454
## [1.7.0] - 2024-10-24
## Added
@@ -192,7 +442,6 @@ and this project adheres to
- 🔥(helm) remove infra related codes #366
## [1.6.0] - 2024-10-17
## Added
@@ -215,7 +464,6 @@ and this project adheres to
- 🐛(backend) fix nginx docker container #340
- 🐛(frontend) fix copy paste firefox #353
## [1.5.1] - 2024-10-10
## Fixed
@@ -250,7 +498,6 @@ and this project adheres to
- 🔧(backend) fix configuration to avoid different ssl warning #297
- 🐛(frontend) fix editor break line not working #302
## [1.4.0] - 2024-09-17
## Added
@@ -271,7 +518,6 @@ and this project adheres to
- 🐛(backend) Fix forcing ID when creating a document via API endpoint #234
- 🐛 Rebuild frontend dev container from makefile #248
## [1.3.0] - 2024-09-05
## Added
@@ -296,7 +542,6 @@ and this project adheres to
- 🔥(frontend) remove saving modal #213
## [1.2.1] - 2024-08-23
## Changed
@@ -304,7 +549,6 @@ and this project adheres to
- ♻️ Change ordering docs datagrid #195
- 🔥(helm) use scaleway email #194
## [1.2.0] - 2024-08-22
## Added
@@ -328,14 +572,14 @@ and this project adheres to
- ⚡️(CI) only e2e chrome mandatory #177
## Removed
- 🔥(helm) remove htaccess #181
- 🔥(helm) remove htaccess #181
## [1.1.0] - 2024-07-15
## Added
- 🤡(demo) generate dummy documents on dev users #120
- 🤡(demo) generate dummy documents on dev users #120
- ✨(frontend) create side modal component #134
- ✨(frontend) Doc grid actions (update / delete) #136
- ✨(frontend) Doc editor header information #137
@@ -346,12 +590,11 @@ and this project adheres to
- ♻️(frontend) create a doc from a modal #132
- ♻️(frontend) manage members from the share modal #140
## [1.0.0] - 2024-07-02
## Added
- 🛂(frontend) Manage the document's right (#75)
- 🛂(frontend) Manage the document's right (#75)
- ✨(frontend) Update document (#68)
- ✨(frontend) Remove document (#68)
- 🐳(docker) dockerize dev frontend (#63)
@@ -374,7 +617,7 @@ and this project adheres to
- ⚡️(e2e) unique login between tests (#80)
- ⚡️(CI) improve e2e job (#86)
- ♻️(frontend) improve the error and message info ui (#93)
- ✏️(frontend) change all occurences of pad to doc (#99)
- ✏️(frontend) change all occurrences of pad to doc (#99)
## Fixed
@@ -385,7 +628,6 @@ and this project adheres to
- 💚(CI) Remove trigger workflow on push tags on CI (#68)
- 🔥(frontend) Remove coming soon page (#121)
## [0.1.0] - 2024-05-24
## Added
@@ -393,8 +635,17 @@ and this project adheres to
- ✨(frontend) Coming Soon page (#67)
- 🚀 Impress, project to manage your documents easily and collaboratively.
[unreleased]: https://github.com/numerique-gouv/impress/compare/v2.1.0...main
[unreleased]: https://github.com/numerique-gouv/impress/compare/v3.3.0...main
[v3.3.0]: https://github.com/numerique-gouv/impress/releases/v3.3.0
[v3.2.1]: https://github.com/numerique-gouv/impress/releases/v3.2.1
[v3.2.0]: https://github.com/numerique-gouv/impress/releases/v3.2.0
[v3.1.0]: https://github.com/numerique-gouv/impress/releases/v3.1.0
[v3.0.0]: https://github.com/numerique-gouv/impress/releases/v3.0.0
[v2.6.0]: https://github.com/numerique-gouv/impress/releases/v2.6.0
[v2.5.0]: https://github.com/numerique-gouv/impress/releases/v2.5.0
[v2.4.0]: https://github.com/numerique-gouv/impress/releases/v2.4.0
[v2.3.0]: https://github.com/numerique-gouv/impress/releases/v2.3.0
[v2.2.0]: https://github.com/numerique-gouv/impress/releases/v2.2.0
[v2.1.0]: https://github.com/numerique-gouv/impress/releases/v2.1.0
[v2.0.1]: https://github.com/numerique-gouv/impress/releases/v2.0.1
[v2.0.0]: https://github.com/numerique-gouv/impress/releases/v2.0.0

View File

@@ -42,34 +42,38 @@ Examples of unacceptable behavior include:
## Enforcement Guidelines
- Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this
- Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of the following Code of Conduct
## Code of Conduct:
1. Correction
### 1. Correction
Community Impact: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
Consequence: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.
2. Warning
### 2. Warning
Community Impact: A violation through a single incident or series of actions.
Consequence: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.
3. Temporary Ban
### 3. Temporary Ban
Community Impact: A serious violation of community standards, including sustained inappropriate behavior.
Consequence: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
4. Permanent Ban
### 4. Permanent Ban
Community Impact: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
Consequence: A permanent ban from any sort of public interaction within the community.
Attribution
## Attribution
This Code of Conduct is adapted from the Contributor Covenant, version 2.1, available at https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.
Community Impact Guidelines were inspired by Mozilla's code of conduct enforcement ladder.
Community Impact Guidelines were inspired by Mozilla's [code of conduct enforcement ladder](https://github.com/mozilla/inclusion/blob/master/code-of-conduct-enforcement/consequence-ladder.md).
For answers to common questions about this code of conduct, see the FAQ at https://www.contributor-covenant.org/faq. Translations are available at https://www.contributor-covenant.org/translations.

View File

@@ -2,14 +2,16 @@
Thank you for taking the time to contribute! Please follow these guidelines to ensure a smooth and productive workflow. 🚀🚀🚀
To get started with the project, please refer to the [README.md](https://github.com/suitenumerique/docs/blob/main/README.md) for detailed instructions.
To get started with the project, please refer to the [README.md](https://github.com/suitenumerique/docs/blob/main/README.md) for detailed instructions on how to run Docs locally.
Contributors are required to sign off their commits with `git commit --signoff`: this confirms that they have read and accepted the [Developer's Certificate of Origin 1.1](https://developercertificate.org/). For security reasons we also require [signing your commits with your SSH or GPG key](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification) with `git commit -S`.
Please also check out our [dev handbook](https://suitenumerique.gitbook.io/handbook) to learn our best practices.
## Help us with translations
You can help us with translations on [Crowdin](https://crowdin.com/project/lasuite-docs).
Your language is not there? Request it on our Crowdin page 😊.
Your language is not there? Request it on our Crowdin page 😊 or ping us on [Matrix](https://matrix.to/#/#docs-official:matrix.org) and let us know if you can help with translations and/or proofreading.
## Creating an Issue
@@ -33,10 +35,14 @@ All commit messages must adhere to the following format:
`<gitmoji>(type) title description`
* <**gitmoji**>: Use a gitmoji to represent the purpose of the commit. For example, ✨ for adding a new feature or 🔥 for removing something, see the list here: <https://gitmoji.dev/>.
* <**gitmoji**>: Use a gitmoji to represent the purpose of the commit. For example, ✨ for adding a new feature or 🔥 for removing something, see the list [here](https://gitmoji.dev/).
* **(type)**: Describe the type of change. Common types include `backend`, `frontend`, `CI`, `docker` etc...
* **title**: A short, descriptive title for the change, starting with a lowercase character.
* **description**: Include additional details about what was changed and why.
* **title**: A short, descriptive title for the change (*)
* **blank line after the commit title
* **description**: Include additional details on why you made the changes (**).
(*) ⚠️ **Make sure you add no space between the emoji and the (type) but add a space after the closing parenthesis of the type and use no caps!**
(**) ⚠️ **Commit description message is mandatory and shouldn't be too long**
### Example Commit Message
@@ -64,7 +70,9 @@ Please add a line to the changelog describing your development. The changelog en
It is nice to add information about the purpose of the pull request to help reviewers understand the context and intent of the changes. If you can, add some pictures or a small video to show the changes.
### Don't forget to:
- check your commits
- signoff your commits
- sign your commits with your key (SSH, GPG etc.)
- check your commits (see warnings above)
- check the linting: `make lint && make frontend-lint`
- check the tests: `make test`
- add a changelog entry
@@ -84,3 +92,11 @@ Make sure that all new features or fixes have corresponding tests. Run the test
If you need any help while contributing, feel free to open a discussion or ask for guidance in the issue tracker. We are more than happy to assist!
Thank you for your contributions! 👍
## Contribute to BlockNote
We use [BlockNote](https://www.blocknotejs.org/) for the text editing features of Docs.
If you find and issue with the editor you can [report it](https://github.com/TypeCellOS/BlockNote/issues) directly on their repository.
Please consider contributing to BlockNotejs, as a library, it's useful to many projects not just Docs.
The project is licended with Mozilla Public License Version 2.0 but be aware that [XL packages](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-docx-exporter/LICENSE) are dual licenced with GNU AFFERO GENERAL PUBLIC LICENCE Version 3 and proprietary licence if you are [sponsor](https://www.blocknotejs.org/pricing).

View File

@@ -1,7 +1,7 @@
# Django impress
# ---- base image to inherit from ----
FROM python:3.12.6-alpine3.20 AS base
FROM python:3.13.3-alpine AS base
# Upgrade pip to its latest release to speed up dependencies installation
RUN python -m pip install --upgrade pip setuptools
@@ -15,6 +15,13 @@ FROM base AS back-builder
WORKDIR /builder
# Install Rust and Cargo using Alpine's package manager
RUN apk add --no-cache \
build-base \
libffi-dev \
rust \
cargo
# Copy required python dependencies
COPY ./src/backend /builder
@@ -23,7 +30,7 @@ RUN mkdir /install && \
# ---- mails ----
FROM node:20 AS mail-builder
FROM node:24 AS mail-builder
COPY ./src/mail /mail/app
@@ -132,6 +139,9 @@ CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
# ---- Production image ----
FROM core AS backend-production
# Remove apk cache, we don't need it anymore
RUN rm -rf /var/cache/apk/*
ARG IMPRESS_STATIC_ROOT=/data/static
# Gunicorn

View File

@@ -39,12 +39,12 @@ DOCKER_UID = $(shell id -u)
DOCKER_GID = $(shell id -g)
DOCKER_USER = $(DOCKER_UID):$(DOCKER_GID)
COMPOSE = DOCKER_USER=$(DOCKER_USER) docker compose
COMPOSE_E2E = DOCKER_USER=$(DOCKER_USER) docker compose -f compose.yml -f compose-e2e.yml
COMPOSE_EXEC = $(COMPOSE) exec
COMPOSE_EXEC_APP = $(COMPOSE_EXEC) app-dev
COMPOSE_RUN = $(COMPOSE) run --rm
COMPOSE_RUN_APP = $(COMPOSE_RUN) app-dev
COMPOSE_RUN_CROWDIN = $(COMPOSE_RUN) crowdin crowdin
WAIT_DB = @$(COMPOSE_RUN) dockerize -wait tcp://$(DB_HOST):$(DB_PORT) -timeout 60s
# -- Backend
MANAGE = $(COMPOSE_RUN_APP) python manage.py
@@ -75,22 +75,39 @@ create-env-files: \
env.d/development/kc_postgresql
.PHONY: create-env-files
bootstrap: ## Prepare Docker images for the project
bootstrap: \
pre-bootstrap: \
data/media \
data/static \
create-env-files \
build \
run-with-frontend \
create-env-files
.PHONY: pre-bootstrap
post-bootstrap: \
migrate \
demo \
back-i18n-compile \
mails-install \
mails-build
.PHONY: post-bootstrap
bootstrap: ## Prepare Docker developmentimages for the project
bootstrap: \
pre-bootstrap \
build \
post-bootstrap \
run
.PHONY: bootstrap
bootstrap-e2e: ## Prepare Docker production images to be used for e2e tests
bootstrap-e2e: \
pre-bootstrap \
build-e2e \
post-bootstrap \
run-e2e
.PHONY: bootstrap-e2e
# -- Docker/compose
build: cache ?= --no-cache
build: cache ?=
build: ## build the project containers
@$(MAKE) build-backend cache=$(cache)
@$(MAKE) build-yjs-provider cache=$(cache)
@@ -104,41 +121,55 @@ build-backend: ## build the app-dev container
build-yjs-provider: cache ?=
build-yjs-provider: ## build the y-provider container
@$(COMPOSE) build y-provider $(cache)
@$(COMPOSE) build y-provider-development $(cache)
.PHONY: build-yjs-provider
build-frontend: cache ?=
build-frontend: ## build the frontend container
@$(COMPOSE) build frontend-dev $(cache)
@$(COMPOSE) build frontend-development $(cache)
.PHONY: build-frontend
build-e2e: cache ?=
build-e2e: ## build the e2e container
@$(MAKE) build-backend cache=$(cache)
@$(COMPOSE_E2E) build frontend $(cache)
@$(COMPOSE_E2E) build y-provider $(cache)
.PHONY: build-e2e
down: ## stop and remove containers, networks, images, and volumes
@$(COMPOSE) down
@$(COMPOSE_E2E) down
.PHONY: down
logs: ## display app-dev logs (follow mode)
@$(COMPOSE) logs -f app-dev
.PHONY: logs
run: ## start the wsgi (production) and development server
run-backend: ## Start only the backend application and all needed services
@$(COMPOSE) up --force-recreate -d celery-dev
@$(COMPOSE) up --force-recreate -d y-provider
@$(COMPOSE) up --force-recreate -d y-provider-development
@$(COMPOSE) up --force-recreate -d nginx
@echo "Wait for postgresql to be up..."
@$(WAIT_DB)
.PHONY: run-backend
run: ## start the wsgi (production) and development server
run:
@$(MAKE) run-backend
@$(COMPOSE) up --force-recreate -d frontend-development
.PHONY: run
run-with-frontend: ## Start all the containers needed (backend to frontend)
@$(MAKE) run
@$(COMPOSE) up --force-recreate -d frontend-dev
.PHONY: run-with-frontend
run-e2e: ## start the e2e server
run-e2e:
@$(MAKE) run-backend
@$(COMPOSE_E2E) stop y-provider-development
@$(COMPOSE_E2E) up --force-recreate -d frontend
@$(COMPOSE_E2E) up --force-recreate -d y-provider
.PHONY: run-e2e
status: ## an alias for "docker compose ps"
@$(COMPOSE) ps
@$(COMPOSE_E2E) ps
.PHONY: status
stop: ## stop the development server using Docker
@$(COMPOSE) stop
@$(COMPOSE_E2E) stop
.PHONY: stop
# -- Backend
@@ -188,14 +219,12 @@ test-back-parallel: ## run all back-end tests in parallel
makemigrations: ## run django makemigrations for the impress project.
@echo "$(BOLD)Running makemigrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(WAIT_DB)
@$(MANAGE) makemigrations
.PHONY: makemigrations
migrate: ## run django migrations for the impress project.
@echo "$(BOLD)Running migrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(WAIT_DB)
@$(MANAGE) migrate
.PHONY: migrate
@@ -310,16 +339,16 @@ help:
.PHONY: help
# Front
frontend-install: ## install the frontend locally
frontend-development-install: ## install the frontend locally
cd $(PATH_FRONT_IMPRESS) && yarn
.PHONY: frontend-install
.PHONY: frontend-development-install
frontend-lint: ## run the frontend linter
cd $(PATH_FRONT) && yarn lint
.PHONY: frontend-lint
run-frontend-development: ## Run the frontend in development mode
@$(COMPOSE) stop frontend-dev
@$(COMPOSE) stop frontend-development
cd $(PATH_FRONT_IMPRESS) && yarn dev
.PHONY: run-frontend-development

125
README.md
View File

@@ -1,13 +1,19 @@
<p align="center">
<a href="https://github.com/suitenumerique/docs">
<img alt="Docs" src="/docs/assets/docs-logo.png" width="300" />
<img alt="Docs" src="/docs/assets/banner-docs.png" width="100%" />
</a>
</p>
<p align="center">
Welcome to Docs! The open source document editor where your notes can become knowledge through live collaboration
<a href="https://github.com/suitenumerique/docs/stargazers/">
<img src="https://img.shields.io/github/stars/suitenumerique/docs" alt="">
</a>
<a href='https://github.com/suitenumerique/docs/blob/main/CONTRIBUTING.md'><img alt='PRs Welcome' src='https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=shields'/></a>
<img alt="GitHub commit activity" src="https://img.shields.io/github/commit-activity/m/suitenumerique/docs"/>
<img alt="GitHub closed issues" src="https://img.shields.io/github/issues-closed/suitenumerique/docs"/>
<a href="https://github.com/suitenumerique/docs/blob/main/LICENSE">
<img alt="MIT License" src="https://img.shields.io/github/license/suitenumerique/docs"/>
</a>
</p>
<p align="center">
<a href="https://matrix.to/#/#docs-official:matrix.org">
Chat on Matrix
@@ -20,76 +26,90 @@ Welcome to Docs! The open source document editor where your notes can become kno
</a>
</p>
# La Suite Docs : Collaborative Text Editing
Docs, where your notes can become knowledge through live collaboration.
<img src="/docs/assets/docs_live_collaboration_light.gif" width="100%" align="center"/>
## Why use Docs ❓
Docs is a collaborative text editor designed to address common challenges in knowledge building and sharing.
### Write
* 😌 Simple collaborative editing without the formatting complexity of markdown
* 🔌 Offline? No problem, keep writing, your edits will get synced when back online
* 💅 Create clean documents with limited but beautiful formatting options and focus on content
* 🧱 Built for productivity (markdown support, many block types, slash commands, keyboard shortcuts).
* ✨ Save time thanks to our AI actions (generate, sum up, correct, translate)
It offers a scalable and secure alternative to tools such as Google Docs, Notion (without the dbs), Outline, or Confluence.
### Collaborate
* 🤝 Collaborate in realtime with your team mates
* 🔒 Granular access control to keep your information secure and shared with the right people
* 📑 Professional document exports in multiple formats (.odt, .doc, .pdf) with customizable templates
* 📚 Built-in wiki functionality to transform your team's collaborative work into organized knowledge `ETA 02/2025`
### Write
* 😌 Get simple, accessible online editing for your team.
* 💅 Create clean documents with beautiful formatting options.
* 🖌️ Focus on your content using either the in-line editor, or [the Markdown syntax](https://www.markdownguide.org/basic-syntax/).
* 🧱 Quickly design your page thanks to the many block types, accessible from the `/` slash commands, as well as keyboard shortcuts.
* 🔌 Write offline! Your edits will be synced once you're back online.
* ✨ Save time thanks to our AI actions, such as rephrasing, summarizing, fixing typos, translating, etc. You can even turn your selected text into a prompt!
### Work together
* 🤝 Enjoy live editing! See your team collaborate in real time.
* 🔒 Keep your information secure thanks to granular access control. Only share with the right people.
* 📑 Export your content in multiple formats (`.odt`, `.docx`, `.pdf`) with customizable templates.
* 📚 Turn your team's collaborative work into organized knowledge with Subpages.
### Self-host
* 🚀 Easy to install, scalable and secure alternative to Notion, Outline or Confluence
🚀 Docs is easy to install on your own servers
Available methods: Helm chart, Nix package
In the works: Docker Compose, YunoHost
⚠️ For some advanced features (ex: Export as PDF) Docs relies on XL packages from BlockNote. These are licenced under AGPL-3.0 and are not MIT compatible. You can perfectly use Docs without these packages by setting the environment variable `PUBLISH_AS_MIT` to true. That way you'll build an image of the application without the features that are not MIT compatible. Read the [environment variables documentation](/docs/env.md) for more information.
## Getting started 🔧
### Test it
Test Docs on your browser by logging in on this [environment](https://impress-preprod.beta.numerique.gouv.fr/docs/0aa856e9-da41-4d59-b73d-a61cb2c1245f/)
```
email: test.docs@yopmail.com
password: I'd<3ToTestDocs
```
### Run it locally
You can test Docs on your browser by visiting this [demo document](https://impress-preprod.beta.numerique.gouv.fr/docs/6ee5aac4-4fb9-457d-95bf-bb56c2467713/)
### Run Docs locally
> ⚠️ The methods described below for running Docs locally is **for testing purposes only**. It is based on building Docs using [Minio](https://min.io/) as an S3-compatible storage solution. Of course you can choose any S3-compatible storage solution.
**Prerequisite**
Make sure you have a recent version of Docker and [Docker Compose](https://docs.docker.com/compose/install) installed on your laptop:
Make sure you have a recent version of Docker and [Docker Compose](https://docs.docker.com/compose/install) installed on your laptop, then type:
```shellscript
$ docker -v
Docker version 20.10.2, build 2291f61
$ docker compose -v
$ docker compose version
docker compose version 1.27.4, build 40524192
Docker Compose version v2.32.4
```
> ⚠️ You may need to run the following commands with sudo but this can be avoided by adding your user to the `docker` group.
> ⚠️ You may need to run the following commands with `sudo`, but this can be avoided by adding your user to the local `docker` group.
**Project bootstrap**
The easiest way to start working on the project is to use GNU Make:
The easiest way to start working on the project is to use [GNU Make](https://www.gnu.org/software/make/):
```shellscript
$ make bootstrap FLUSH_ARGS='--no-input'
```
This command builds the `app` container, installs dependencies, performs database migrations and compile translations. It's a good idea to use this
command each time you are pulling code from the project repository to avoid dependency-related or migration-related issues.
This command builds the `app-dev` and `frontend-dev` containers, installs dependencies, performs database migrations and compiles translations. It's a good idea to use this command each time you are pulling code from the project repository to avoid dependency-related or migration-related issues.
Your Docker services should now be up and running 🎉
You can access to the project by going to <http://localhost:3000>.
You can access the project by going to <http://localhost:3000>.
You will be prompted to log in, the default credentials are:
You will be prompted to log in. The default credentials are:
```
username: impress
password: impress
```
📝 Note that if you need to run them afterwards, you can use the eponym Make rule:
📝 Note that if you need to run them afterwards, you can use the eponymous Make rule:
```shellscript
$ make run-with-frontend
$ make run
```
⚠️ For the frontend developer, it is often better to run the frontend in development mode locally.
@@ -97,7 +117,7 @@ $ make run-with-frontend
To do so, install the frontend dependencies with the following command:
```shellscript
$ make frontend-install
$ make frontend-development-install
```
And run the frontend locally in development mode with the following command:
@@ -109,24 +129,26 @@ $ make run-frontend-development
To start all the services, except the frontend container, you can use the following command:
```shellscript
$ make run
$ make run-backend
```
**Adding content**
You can create a basic demo site by running:
You can create a basic demo site by running this command:
```shellscript
$ make demo
```
Finally, you can check all available Make rules using:
Finally, you can check all available Make rules using this command:
```shellscript
$ make help
```
**Django admin**
You can access the Django admin site at
You can access the Django admin site at:
<http://localhost:8071/admin>.
@@ -137,24 +159,28 @@ $ make superuser
```
## Feedback 🙋‍♂️🙋‍♀️
We'd love to hear your thoughts and hear about your experiments, so come and say hi on [Matrix](https://matrix.to/#/#docs-official:matrix.org).
## Roadmap
We'd love to hear your thoughts, and hear about your experiments, so come and say hi on [Matrix](https://matrix.to/#/#docs-official:matrix.org).
## Roadmap 💡
Want to know where the project is headed? [🗺️ Checkout our roadmap](https://github.com/orgs/numerique-gouv/projects/13/views/11)
## Licence 📝
## License 📝
This work is released under the MIT License (see [LICENSE](https://github.com/suitenumerique/docs/blob/main/LICENSE)).
While Docs is a public driven initiative our licence choice is an invitation for private sector actors to use, sell and contribute to the project.
While Docs is a public-driven initiative, our license choice is an invitation for private sector actors to use, sell and contribute to the project.
## Contributing 🙌
This project is intended to be community-driven, so please, do not hesitate to [get in touch](https://matrix.to/#/#docs-official:matrix.org) if you have any question related to our implementation or design decisions.
You can help us with translations on [Crowdin](https://crowdin.com/project/lasuite-docs).
If you intend to make pull requests see [CONTRIBUTING](https://github.com/suitenumerique/docs/blob/main/CONTRIBUTING.md) for guidelines.
If you intend to make pull requests, see [CONTRIBUTING](https://github.com/suitenumerique/docs/blob/main/CONTRIBUTING.md) for guidelines.
Directory structure:
## Directory structure:
```markdown
docs
@@ -169,14 +195,17 @@ docs
```
## Credits ❤️
### Stack
Docs is built on top of [Django Rest Framework](https://www.django-rest-framework.org/), [Next.js](https://nextjs.org/), [MinIO](https://min.io/), [BlockNote.js](https://www.blocknotejs.org/), [HocusPocus](https://tiptap.dev/docs/hocuspocus/introduction) and [Yjs](https://yjs.dev/)
Docs is built on top of [Django Rest Framework](https://www.django-rest-framework.org/), [Next.js](https://nextjs.org/), [BlockNote.js](https://www.blocknotejs.org/), [HocusPocus](https://tiptap.dev/docs/hocuspocus/introduction) and [Yjs](https://yjs.dev/). We thank the contributors of all these projects for their awesome work!
We are proud sponsors of [BlockNotejs](https://www.blocknotejs.org/) and [Yjs](https://yjs.dev/).
### Gov ❤️ open source
Docs is the result of a joint effort led by the French 🇫🇷🥖 ([DINUM](https://www.numerique.gouv.fr/dinum/)) and German 🇩🇪🥨 governments ([ZenDiS](https://zendis.de/)).
We are proud sponsors of [BlockNotejs](https://www.blocknotejs.org/) and [Yjs](https://yjs.dev/).
We are always looking for new public partners (we are currently onboarding the Netherlands 🇳🇱🧀), feel free to [reach out](mailto:docs@numerique.gouv.fr) if you are interested in using or contributing to Docs.
<p align="center">

View File

@@ -4,7 +4,7 @@
Security is very important to us.
If you have any issue regarding security, please disclose the information responsibly submiting [this form](https://vdp.numerique.gouv.fr/p/Send-a-report?lang=en) and not by creating an issue on the repository. You can also email us at docs@numerique.gouv.fr
If you have any issue regarding security, please disclose the information responsibly submitting [this form](https://vdp.numerique.gouv.fr/p/Send-a-report?lang=en) and not by creating an issue on the repository. You can also email us at docs@numerique.gouv.fr
We appreciate your effort to make Docs more secure.

View File

@@ -15,3 +15,29 @@ the following command inside your docker container:
(Note : in your development environment, you can `make migrate`.)
## [Unreleased]
## [3.3.0] - 2025-05-22
⚠️ For some advanced features (ex: Export as PDF) Docs relies on XL packages from BlockNote. These are licenced under AGPL-3.0 and are not MIT compatible. You can perfectly use Docs without these packages by setting the environment variable `PUBLISH_AS_MIT` to true. That way you'll build an image of the application without the features that are not MIT compatible. Read the [environment variables documentation](/docs/env.md) for more information.
The footer is now configurable from a customization file. To override the default one, you can
use the `THEME_CUSTOMIZATION_FILE_PATH` environment variable to point to your customization file.
The customization file must be a JSON file and must follow the rules described in the
[theming documentation](docs/theming.md).
## [3.0.0] - 2025-03-28
We are not using the nginx auth request anymore to access the collaboration server (`yProvider`)
The authentication is now managed directly from the yProvider server.
You must remove the annotation `nginx.ingress.kubernetes.io/auth-url` from the `ingressCollaborationWS`.
This means as well that the yProvider server must be able to access the Django server.
To do so, you must set the `COLLABORATION_BACKEND_BASE_URL` environment variable to the `yProvider`
service.
## [2.2.0] - 2025-02-10
- AI features are now limited to users who are authenticated. Before this release, even anonymous
users who gained editor access on a document with link reach used to get AI feature.
IF you want anonymous users to keep access on AI features, you must now define the
`AI_ALLOW_REACH_FROM` setting to "public".

View File

@@ -39,6 +39,9 @@ docker_build(
]
)
k8s_resource('impress-docs-backend-migrate', resource_deps=['postgres-postgresql'])
k8s_resource('impress-docs-backend-createsuperuser', resource_deps=['impress-docs-backend-migrate'])
k8s_resource('impress-docs-backend', resource_deps=['impress-docs-backend-migrate'])
k8s_yaml(local('cd ../src/helm && helmfile -n impress -e dev template .'))
migration = '''

View File

@@ -6,8 +6,7 @@ REPO_DIR="$(cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd)"
UNSET_USER=0
TERRAFORM_DIRECTORY="./env.d/terraform"
COMPOSE_FILE="${REPO_DIR}/docker-compose.yml"
COMPOSE_PROJECT="docs"
COMPOSE_FILE="${REPO_DIR}/compose.yml"
# _set_user: set (or unset) default user id used to run docker commands
@@ -40,9 +39,8 @@ function _set_user() {
# ARGS : docker compose command arguments
function _docker_compose() {
echo "🐳(compose) project: '${COMPOSE_PROJECT}' file: '${COMPOSE_FILE}'"
echo "🐳(compose) file: '${COMPOSE_FILE}'"
docker compose \
-p "${COMPOSE_PROJECT}" \
-f "${COMPOSE_FILE}" \
--project-directory "${REPO_DIR}" \
"$@"

28
compose-e2e.yml Normal file
View File

@@ -0,0 +1,28 @@
services:
frontend:
user: "${DOCKER_USER:-1000}"
build:
context: .
dockerfile: ./src/frontend/Dockerfile
target: frontend-production
args:
API_ORIGIN: "http://localhost:8071"
PUBLISH_AS_MIT: "false"
SW_DEACTIVATED: "true"
image: impress:frontend-production
ports:
- "3000:3000"
y-provider:
user: ${DOCKER_USER:-1000}
build:
context: .
dockerfile: ./src/frontend/servers/y-provider/Dockerfile
target: y-provider
image: impress:y-provider-production
restart: unless-stopped
env_file:
- env.d/development/common
ports:
- "4444:4444"

View File

@@ -1,6 +1,13 @@
name: docs
services:
postgresql:
image: postgres:16
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 1s
timeout: 2s
retries: 300
env_file:
- env.d/development/postgresql
ports:
@@ -23,6 +30,11 @@ services:
ports:
- '9000:9000'
- '9001:9001'
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 1s
timeout: 20s
retries: 300
entrypoint: ""
command: minio server --console-address :9001 /data
volumes:
@@ -31,7 +43,9 @@ services:
createbuckets:
image: minio/mc
depends_on:
- minio
minio:
condition: service_healthy
restart: true
entrypoint: >
sh -c "
/usr/bin/mc alias set impress http://minio:9000 impress password && \
@@ -59,10 +73,15 @@ services:
- ./src/backend:/app
- ./data/static:/data/static
depends_on:
- postgresql
- mailcatcher
- redis
- createbuckets
postgresql:
condition: service_healthy
restart: true
mailcatcher:
condition: service_started
redis:
condition: service_started
createbuckets:
condition: service_started
celery-dev:
user: ${DOCKER_USER:-1000}
@@ -79,36 +98,6 @@ services:
depends_on:
- app-dev
app:
build:
context: .
target: backend-production
args:
DOCKER_USER: ${DOCKER_USER:-1000}
user: ${DOCKER_USER:-1000}
image: impress:backend-production
environment:
- DJANGO_CONFIGURATION=Demo
env_file:
- env.d/development/common
- env.d/development/postgresql
depends_on:
- postgresql
- redis
- minio
celery:
user: ${DOCKER_USER:-1000}
image: impress:backend-production
command: ["celery", "-A", "impress.celery_app", "worker", "-l", "INFO"]
environment:
- DJANGO_CONFIGURATION=Demo
env_file:
- env.d/development/common
- env.d/development/postgresql
depends_on:
- app
nginx:
image: nginx:1.25
ports:
@@ -116,28 +105,30 @@ services:
volumes:
- ./docker/files/etc/nginx/conf.d:/etc/nginx/conf.d:ro
depends_on:
- keycloak
- app-dev
- y-provider
app-dev:
condition: service_started
keycloak:
condition: service_healthy
restart: true
frontend-dev:
frontend-development:
user: "${DOCKER_USER:-1000}"
build:
context: .
dockerfile: ./src/frontend/Dockerfile
target: frontend-production
target: impress-dev
args:
API_ORIGIN: "http://localhost:8071"
Y_PROVIDER_URL: "ws://localhost:4444"
MEDIA_URL: "http://localhost:8083"
PUBLISH_AS_MIT: "false"
SW_DEACTIVATED: "true"
image: impress:frontend-development
volumes:
- ./src/frontend:/home/frontend
- /home/frontend/node_modules
- /home/frontend/apps/impress/node_modules
ports:
- "3000:3000"
dockerize:
image: jwilder/dockerize
crowdin:
image: crowdin/cli:3.16.0
volumes:
@@ -148,27 +139,37 @@ services:
working_dir: /app
node:
image: node:18
image: node:22
user: "${DOCKER_USER:-1000}"
environment:
HOME: /tmp
volumes:
- ".:/app"
y-provider:
y-provider-development:
user: ${DOCKER_USER:-1000}
build:
context: .
dockerfile: ./src/frontend/servers/y-provider/Dockerfile
target: y-provider
target: y-provider-development
image: impress:y-provider-development
restart: unless-stopped
env_file:
- env.d/development/common
ports:
- "4444:4444"
volumes:
- ./src/frontend/:/home/frontend
- /home/frontend/node_modules
- /home/frontend/servers/y-provider/node_modules
kc_postgresql:
image: postgres:14.3
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 1s
timeout: 2s
retries: 300
ports:
- "5433:5432"
env_file:
@@ -187,6 +188,13 @@ services:
- --hostname-admin-url=http://localhost:8083/
- --hostname-strict=false
- --hostname-strict-https=false
- --health-enabled=true
- --metrics-enabled=true
healthcheck:
test: ["CMD", "curl", "--head", "-fsS", "http://localhost:8080/health/ready"]
interval: 1s
timeout: 2s
retries: 300
environment:
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: admin
@@ -200,4 +208,6 @@ services:
ports:
- "8080:8080"
depends_on:
- kc_postgresql
kc_postgresql:
condition: service_healthy
restart: true

View File

@@ -60,7 +60,7 @@
},
{
"username": "user-e2e-chromium",
"email": "user@chromium.e2e",
"email": "user@chromium.test",
"firstName": "E2E",
"lastName": "Chromium",
"enabled": true,
@@ -74,7 +74,7 @@
},
{
"username": "user-e2e-webkit",
"email": "user@webkit.e2e",
"email": "user@webkit.test",
"firstName": "E2E",
"lastName": "Webkit",
"enabled": true,
@@ -88,7 +88,7 @@
},
{
"username": "user-e2e-firefox",
"email": "user@firefox.e2e",
"email": "user@firefox.test",
"firstName": "E2E",
"lastName": "Firefox",
"enabled": true,

View File

@@ -4,54 +4,6 @@ server {
server_name localhost;
charset utf-8;
# Proxy auth for collaboration server
location /collaboration/ws/ {
# Collaboration Auth request configuration
auth_request /collaboration-auth;
auth_request_set $authHeader $upstream_http_authorization;
auth_request_set $canEdit $upstream_http_x_can_edit;
auth_request_set $userId $upstream_http_x_user_id;
# Pass specific headers from the auth response
proxy_set_header Authorization $authHeader;
proxy_set_header X-Can-Edit $canEdit;
proxy_set_header X-User-Id $userId;
# Ensure WebSocket upgrade
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
# Collaboration server
proxy_pass http://y-provider:4444;
# Set appropriate timeout for WebSocket
proxy_read_timeout 86400;
proxy_send_timeout 86400;
# Preserve original host and additional headers
proxy_set_header Host $host;
}
location /collaboration-auth {
proxy_pass http://app-dev:8000/api/v1.0/documents/collaboration-auth/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Original-URL $request_uri;
# Prevent the body from being passed
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-Method $request_method;
}
location /collaboration/api/ {
# Collaboration server
proxy_pass http://y-provider:4444;
proxy_set_header Host $host;
}
# Proxy auth for media
location /media/ {
# Auth request configuration
@@ -68,6 +20,8 @@ server {
# Get resource from Minio
proxy_pass http://minio:9000/impress-media-storage/;
proxy_set_header Host minio:9000;
add_header Content-Security-Policy "default-src 'none'" always;
}
location /media-auth {
@@ -88,5 +42,11 @@ server {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Increase proxy buffer size to allow keycloak to send large
# header responses when a user is created.
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
}
}

View File

@@ -0,0 +1,112 @@
upstream docs_backend {
server ${BACKEND_HOST}:8000 fail_timeout=0;
}
upstream docs_frontend {
server ${FRONTEND_HOST}:3000 fail_timeout=0;
}
server {
listen 8083;
server_name localhost;
charset utf-8;
# Disables server version feedback on pages and in headers
server_tokens off;
proxy_ssl_server_name on;
location @proxy_to_docs_backend {
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
proxy_pass http://docs_backend;
}
location @proxy_to_docs_frontend {
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
proxy_pass http://docs_frontend;
}
location / {
try_files $uri @proxy_to_docs_frontend;
}
location /api {
try_files $uri @proxy_to_docs_backend;
}
location /admin {
try_files $uri @proxy_to_docs_backend;
}
location /static {
try_files $uri @proxy_to_docs_backend;
}
# Proxy auth for collaboration server
location /collaboration/ws/ {
# Ensure WebSocket upgrade
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
# Collaboration server
proxy_pass http://${YPROVIDER_HOST}:4444;
# Set appropriate timeout for WebSocket
proxy_read_timeout 86400;
proxy_send_timeout 86400;
# Preserve original host and additional headers
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Origin $http_origin;
proxy_set_header Host $host;
}
location /collaboration/api/ {
# Collaboration server
proxy_pass http://${YPROVIDER_HOST}:4444;
proxy_set_header Host $host;
}
# Proxy auth for media
location /media/ {
# Auth request configuration
auth_request /media-auth;
auth_request_set $authHeader $upstream_http_authorization;
auth_request_set $authDate $upstream_http_x_amz_date;
auth_request_set $authContentSha256 $upstream_http_x_amz_content_sha256;
# Pass specific headers from the auth response
proxy_set_header Authorization $authHeader;
proxy_set_header X-Amz-Date $authDate;
proxy_set_header X-Amz-Content-SHA256 $authContentSha256;
# Get resource from Minio
proxy_pass https://${S3_HOST}/${BUCKET_NAME}/;
proxy_set_header Host ${S3_HOST};
proxy_ssl_name ${S3_HOST};
add_header Content-Security-Policy "default-src 'none'" always;
}
location /media-auth {
proxy_pass http://docs_backend/api/v1.0/documents/media-auth/;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Original-URL $request_uri;
# Prevent the body from being passed
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-Method $request_method;
}
}

View File

@@ -0,0 +1,193 @@
## Decision TLDR;
We will use Yjs a CRDT-based library for the collaborative editing of the documents.
## Status
Accepted
## Context
We need to implement a collaborative editing feature for the documents that supports real-time collaboration, offline capabilities, and seamless integration with our Django backend.
## Considered alternatives
### ProseMirror
A robust toolkit for building rich-text editors with collaboration capabilities.
| Pros | Cons |
| --- | --- |
| Mature ecosystem | Complex integration with Django |
| Rich text editing features | Steeper learning curve |
| Used by major companies | More complex to implement offline support |
| Large community | |
### ShareDB
Real-time database backend based on Operational Transformation.
| Pros | Cons |
| --- | --- |
| Battle-tested in production | Complex setup required |
| Strong consistency model | Requires specific backend architecture |
| Good documentation | Less flexible with different backends |
| | Higher latency compared to CRDTs |
### Convergence
Complete enterprise solution for real-time collaboration.
| Pros | Cons |
| --- | --- |
| Full-featured solution | Commercial licensing |
| Built-in presence features | Less community support |
| Enterprise support | More expensive |
| Good offline support | Overkill for basic needs |
### CRDT-based Solutions Comparison
A CRDT-based library specifically designed for real-time collaboration.
| Category | Pros | Cons |
|----------|------|------|
| Technical Implementation | • Native real-time collaboration<br>• No central conflict resolution needed<br>• Works well with Django backend<br>• Automatic state synchronization | • Learning curve for CRDT concepts<br>• More complex initial setup<br>• Additional metadata overhead |
| User Experience | • Instant local updates<br>• Works offline by default<br>• Low latency<br>• Smooth concurrent editing | • Eventual consistency might cause brief inconsistencies<br>• UI must handle temporary conflicts |
| Performance | • Excellent scaling with multiple users<br>• Reduced server load<br>• Efficient network usage<br>• Good memory optimization (especially Yjs) | • Slightly higher memory usage<br>• Initial state sync can be larger |
| Development | • No need to build conflict resolution<br>• Simple integration with text editors<br>• Future-proof architecture | • Team needs to learn new concepts<br>• Fewer ready-made solutions<br>• May need to build some features from scratch |
| Maintenance | • Less server infrastructure<br>• Simpler deployment<br>• Fewer points of failure | • Debugging can be more complex<br>• State management requires careful handling |
| Business Impact | • Better offline support for users<br>• Scales well as user base grows<br>• No licensing costs (with Yjs) | • Initial development time might be longer<br>• Team training required |
#### Yjs
- **Type**: State-based CRDT
- **Implementation**: JavaScript/TypeScript
- **Features**:
- Rich text collaboration
- Shared types (Array, Map, XML)
- Binary encoding
- P2P support
- **Performance**: Excellent for text editing
- **Memory Usage**: Optimized
- **License**: MIT
#### Automerge
- **Type**: Operation-based CRDT
- **Implementation**: JavaScript/Rust
- **Features**:
- JSON-like data structures
- Change history
- Undo/Redo
- Binary format
- **Performance**: Good, with Rust backend
- **Memory Usage**: Higher than Yjs
- **License**: MIT
#### Legion
- **Type**: State-based CRDT
- **Implementation**: Rust with JS bindings
- **Features**:
- High performance
- Memory efficient
- Binary protocol
- **Performance**: Excellent
- **Memory Usage**: Very efficient
- **License**: Apache 2.0
#### Diamond Types
- **Type**: Operation-based CRDT
- **Implementation**: TypeScript
- **Features**:
- Specialized for text
- Small memory footprint
- Simple API
- **Performance**: Good for text
- **Memory Usage**: Efficient
- **License**: MIT
Comparison Table:
| Feature | Yjs | Automerge | Legion | Diamond Types |
|---------|-----|-----------|--------|---------------|
| Text Editing | ✅ Excellent | ✅ Good | ⚠️ Basic | ✅ Excellent |
| Structured Data | ✅ | ✅ | ✅ | ⚠️ |
| Memory Efficiency | ✅ High | ⚠️ Medium | ✅ Very High | ✅ High |
| Network Efficiency | ✅ | ⚠️ | ✅ | ✅ |
| Maturity | ✅ | ✅ | ⚠️ | ⚠️ |
| Community Size | ✅ Large | ✅ Large | ⚠️ Small | ⚠️ Small |
| Documentation | ✅ | ✅ | ⚠️ | ⚠️ |
| Backend Options | ✅ Many | ✅ Many | ⚠️ Limited | ⚠️ Limited |
Key Differences:
1. **Implementation Approach**:
- Yjs: Optimized for text and rich-text editing
- Automerge: General-purpose JSON CRDT
- Legion: Performance-focused with Rust
- Diamond Types: Specialized for text collaboration
2. **Performance Characteristics**:
- Yjs: Best for text editing scenarios
- Automerge: Good all-around performance
- Legion: Excellent raw performance
- Diamond Types: Optimized for text
3. **Ecosystem Integration**:
- Yjs: Wide range of integrations
- Automerge: Good JavaScript ecosystem
- Legion: Limited but growing
- Diamond Types: Focused on text editors
This analysis reinforces our choice of Yjs for the CRDT-based option as it provides:
- Best-in-class text editing performance
- Mature ecosystem
- Active community
- Excellent documentation
- Wide range of backend options
## Decision
After evaluating the alternatives, we choose Yjs for the following reasons:
1. **Technical Fit:**
- Native CRDT support ensures reliable collaboration
- Excellent offline capabilities
- Good performance characteristics
- Flexible backend integration options
2. **Project Requirements Match:**
- Easy integration with our Django backend
- Supports our core collaborative features
- Manageable learning curve for the team
3. **Community & Support:**
- Active development
- Growing community
- Good documentation
- Open source with MIT license
### Comparison of Key Features:
| Feature | Yjs (CRDT) | ProseMirror | ShareDB | Convergence |
|---------|-----|-------------|----------|-------------|
| Real-time Collaboration | ✅ | ✅ | ✅ | ✅ |
| Offline Support | ✅ | ⚠️ | ⚠️ | ✅ |
| Django Integration | Easy | Complex | Complex | Moderate |
| Learning Curve | Medium | High | High | Medium |
| Cost | Free | Free | Free | Paid |
| Community Size | Growing | Large | Medium | Small |
## Consequences
### Positive
- Simplified implementation of real-time collaboration
- Good developer experience
- Future-proof technology choice
- No licensing costs
### Negative
- Team needs to learn CRDT concepts
- Newer technology compared to alternatives
- May need to build some features available out-of-the-box in other solutions
### Risks
- Community support might not grow as expected
- May discover limitations as we scale

19
docs/architecture.md Normal file
View File

@@ -0,0 +1,19 @@
## Architecture
### Global system architecture
```mermaid
flowchart TD
User -- HTTP --> Front("Frontend (NextJS SPA)")
Front -- REST API --> Back("Backend (Django)")
Front -- WebSocket --> Yserver("Microservice Yjs (Express)") -- WebSocket --> CollaborationServer("Collaboration server (Hocuspocus)") -- REST API <--> Back
Front -- OIDC --> Back -- OIDC ---> OIDC("Keycloak / ProConnect")
Back -- REST API --> Yserver
Back --> DB("Database (PostgreSQL)")
Back <--> Celery --> DB
Back ----> S3("Minio (S3)")
```
### Architecture decision records
- [ADR-0001-20250106-use-yjs-for-docs-editing](./adr/ADR-0001-20250106-use-yjs-for-docs-editing.md)

BIN
docs/assets/banner-docs.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 215 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

144
docs/env.md Normal file
View File

@@ -0,0 +1,144 @@
# Docs variables
Here we describe all environment variables that can be set for the docs application.
## impress-backend container
These are the environment variables you can set for the `impress-backend` container.
| Option | Description | default |
| ----------------------------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------- |
| AI_ALLOW_REACH_FROM | Users that can use AI must be this level. options are "public", "authenticated", "restricted" | authenticated |
| AI_API_KEY | AI key to be used for AI Base url | |
| AI_BASE_URL | OpenAI compatible AI base url | |
| AI_FEATURE_ENABLED | Enable AI options | false |
| AI_MODEL | AI Model to use | |
| ALLOW_LOGOUT_GET_METHOD | Allow get logout method | true |
| API_USERS_LIST_LIMIT | Limit on API users | 5 |
| API_USERS_LIST_THROTTLE_RATE_BURST | throttle rate for api on burst | 30/minute |
| API_USERS_LIST_THROTTLE_RATE_SUSTAINED | throttle rate for api | 180/hour |
| AWS_S3_ACCESS_KEY_ID | access id for s3 endpoint | |
| AWS_S3_ENDPOINT_URL | S3 endpoint | |
| AWS_S3_REGION_NAME | region name for s3 endpoint | |
| AWS_S3_SECRET_ACCESS_KEY | access key for s3 endpoint | |
| AWS_STORAGE_BUCKET_NAME | bucket name for s3 endpoint | impress-media-storage |
| CACHES_DEFAULT_TIMEOUT | cache default timeout | 30 |
| CACHES_KEY_PREFIX | The prefix used to every cache keys. | docs |
| COLLABORATION_API_URL | collaboration api host | |
| COLLABORATION_SERVER_SECRET | collaboration api secret | |
| COLLABORATION_WS_NOT_CONNECTED_READY_ONLY | Users not connected to the collaboration server cannot edit | false |
| COLLABORATION_WS_URL | collaboration websocket url | |
| CONVERSION_API_CONTENT_FIELD | Conversion api content field | content |
| CONVERSION_API_ENDPOINT | Conversion API endpoint | convert-markdown |
| CONVERSION_API_SECURE | Require secure conversion api | false |
| CONVERSION_API_TIMEOUT | Conversion api timeout | 30 |
| CONTENT_SECURITY_POLICY_DIRECTIVES | A dict of directives set in the Content-Security-Policy header | All directives are set to 'none' |
| CONTENT_SECURITY_POLICY_EXCLUDE_URL_PREFIXES | Url with this prefix will not have the header Content-Security-Policy included | |
| CRISP_WEBSITE_ID | crisp website id for support | |
| DB_ENGINE | engine to use for database connections | django.db.backends.postgresql_psycopg2 |
| DB_HOST | host of the database | localhost |
| DB_NAME | name of the database | impress |
| DB_PASSWORD | password to authenticate with | pass |
| DB_PORT | port of the database | 5432 |
| DB_USER | user to authenticate with | dinum |
| DJANGO_ALLOWED_HOSTS | allowed hosts | [] |
| DJANGO_CELERY_BROKER_TRANSPORT_OPTIONS | celery broker transport options | {} |
| DJANGO_CELERY_BROKER_URL | celery broker url | redis://redis:6379/0 |
| DJANGO_CORS_ALLOW_ALL_ORIGINS | allow all CORS origins | false |
| DJANGO_CORS_ALLOWED_ORIGIN_REGEXES | list of origins allowed for CORS using regulair expressions | [] |
| DJANGO_CORS_ALLOWED_ORIGINS | list of origins allowed for CORS | [] |
| DJANGO_CSRF_TRUSTED_ORIGINS | CSRF trusted origins | [] |
| DJANGO_EMAIL_BACKEND | email backend library | django.core.mail.backends.smtp.EmailBackend |
| DJANGO_EMAIL_BRAND_NAME | brand name for email | |
| DJANGO_EMAIL_FROM | email address used as sender | from@example.com |
| DJANGO_EMAIL_HOST | host name of email | |
| DJANGO_EMAIL_HOST_PASSWORD | password to authenticate with on the email host | |
| DJANGO_EMAIL_HOST_USER | user to authenticate with on the email host | |
| DJANGO_EMAIL_LOGO_IMG | logo for the email | |
| DJANGO_EMAIL_PORT | port used to connect to email host | |
| DJANGO_EMAIL_USE_SSL | use sstl for email host connection | false |
| DJANGO_EMAIL_USE_TLS | use tls for email host connection | false |
| DJANGO_SECRET_KEY | secret key | |
| DJANGO_SERVER_TO_SERVER_API_TOKENS | | [] |
| DOCUMENT_IMAGE_MAX_SIZE | maximum size of document in bytes | 10485760 |
| FRONTEND_CSS_URL | To add a external css file to the app | |
| FRONTEND_HOMEPAGE_FEATURE_ENABLED | frontend feature flag to display the homepage | false |
| FRONTEND_THEME | frontend theme to use | |
| LANGUAGE_CODE | default language | en-us |
| LOGGING_LEVEL_LOGGERS_APP | application logging level. options are "DEBUG", "INFO", "WARN", "ERROR", "CRITICAL" | INFO |
| LOGGING_LEVEL_LOGGERS_ROOT | default logging level. options are "DEBUG", "INFO", "WARN", "ERROR", "CRITICAL" | INFO |
| LOGIN_REDIRECT_URL | login redirect url | |
| LOGIN_REDIRECT_URL_FAILURE | login redirect url on failure | |
| LOGOUT_REDIRECT_URL | logout redirect url | |
| MALWARE_DETECTION_BACKEND | The malware detection backend use from the django-lasuite package | lasuite.malware_detection.backends.dummy.DummyBackend |
| MALWARE_DETECTION_PARAMETERS | A dict containing all the parameters to initiate the malware detection backend | {"callback_path": "core.malware_detection.malware_detection_callback",} |
| MEDIA_BASE_URL | | |
| OIDC_ALLOW_DUPLICATE_EMAILS | Allow duplicate emails | false |
| OIDC_AUTH_REQUEST_EXTRA_PARAMS | OIDC extra auth parameters | {} |
| OIDC_CREATE_USER | create used on OIDC | false |
| OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION | faillback to email for identification | true |
| OIDC_OP_AUTHORIZATION_ENDPOINT | Authorization endpoint for OIDC | |
| OIDC_OP_JWKS_ENDPOINT | JWKS endpoint for OIDC | |
| OIDC_OP_LOGOUT_ENDPOINT | Logout endpoint for OIDC | |
| OIDC_OP_TOKEN_ENDPOINT | Token endpoint for OIDC | |
| OIDC_OP_USER_ENDPOINT | User endpoint for OIDC | |
| OIDC_REDIRECT_ALLOWED_HOSTS | Allowed hosts for OIDC redirect url | [] |
| OIDC_REDIRECT_REQUIRE_HTTPS | Require https for OIDC redirect url | false |
| OIDC_RP_CLIENT_ID | client id used for OIDC | impress |
| OIDC_RP_CLIENT_SECRET | client secret used for OIDC | |
| OIDC_RP_SCOPES | scopes requested for OIDC | openid email |
| OIDC_RP_SIGN_ALGO | verification algorithm used OIDC tokens | RS256 |
| OIDC_STORE_ID_TOKEN | Store OIDC token | true |
| OIDC_USE_NONCE | use nonce for OIDC | true |
| OIDC_USERINFO_FULLNAME_FIELDS | OIDC token claims to create full name | ["first_name", "last_name"] |
| OIDC_USERINFO_SHORTNAME_FIELD | OIDC token claims to create shortname | first_name |
| POSTHOG_KEY | posthog key for analytics | |
| REDIS_URL | cache url | redis://redis:6379/1 |
| SENTRY_DSN | sentry host | |
| SESSION_COOKIE_AGE | duration of the cookie session | 60*60*12 |
| SPECTACULAR_SETTINGS_ENABLE_DJANGO_DEPLOY_CHECK | | false |
| STORAGES_STATICFILES_BACKEND | | whitenoise.storage.CompressedManifestStaticFilesStorage |
| THEME_CUSTOMIZATION_CACHE_TIMEOUT | Cache duration for the customization settings | 86400 |
| THEME_CUSTOMIZATION_FILE_PATH | full path to the file customizing the theme. An example is provided in src/backend/impress/configuration/theme/default.json | BASE_DIR/impress/configuration/theme/default.json |
| TRASHBIN_CUTOFF_DAYS | trashbin cutoff | 30 |
| USER_OIDC_ESSENTIAL_CLAIMS | essential claims in OIDC token | [] |
| Y_PROVIDER_API_BASE_URL | Y Provider url | |
| Y_PROVIDER_API_KEY | Y provider API key | |
## impress-frontend image
These are the environment variables you can set to build the `impress-frontend` image.
Depending on how you are building the front-end application, this variable is used in different ways.
If you want to build the Docker image, this variable is used as an argument in the build command.
Example:
```
docker build -f src/frontend/Dockerfile --target frontend-production --build-arg PUBLISH_AS_MIT=false docs-frontend:latest
```
If you want to build the front-end application using the yarn build command, you can edit the file `src/frontend/apps/impress/.env` with the `NODE_ENV=production` environment variable and modify it. Alternatively, you can use the listed environment variables with the prefix `NEXT_PUBLIC_` (for example, `NEXT_PUBLIC_PUBLISH_AS_MIT=false`).
Example:
```
cd src/frontend/apps/impress
NODE_ENV=production NEXT_PUBLIC_PUBLISH_AS_MIT=false yarn build
```
| Option | Description | default |
| ----------------------------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------- |
| API_ORIGIN | backend domain - it uses the current domain if not initialized | |
| SW_DEACTIVATED | To not install the service worker | |
| PUBLISH_AS_MIT | Removes packages whose licences are incompatible with the MIT licence (see below) | true |
Packages with licences incompatible with the MIT licence:
* `xl-docx-exporter`: [AGPL-3.0](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-docx-exporter/LICENSE),
* `xl-pdf-exporter`: [AGPL-3.0](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-pdf-exporter/LICENSE)
In `.env.development`, `PUBLISH_AS_MIT` is set to `false`, allowing developers to test Docs with all its features.
⚠️ If you run Docs in production with `PUBLISH_AS_MIT` set to `false` make sure you fulfill your [BlockNote licensing](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-pdf-exporter/LICENSE) or [subscription](https://www.blocknotejs.org/about#partner-with-us) obligations.

View File

@@ -0,0 +1,78 @@
services:
postgresql:
image: postgres:16
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 1s
timeout: 2s
retries: 300
env_file:
- env.d/postgresql
- env.d/common
environment:
- PGDATA=/var/lib/postgresql/data/pgdata
volumes:
- ./data/databases/backend:/var/lib/postgresql/data/pgdata
redis:
image: redis:8
backend:
image: lasuite/impress-backend:latest
user: ${DOCKER_USER:-1000}
restart: always
environment:
- DJANGO_CONFIGURATION=Production
env_file:
- env.d/common
- env.d/backend
- env.d/yprovider
- env.d/postgresql
healthcheck:
test: ["CMD", "python", "manage.py", "check"]
interval: 15s
timeout: 30s
retries: 20
start_period: 10s
depends_on:
postgresql:
condition: service_healthy
restart: true
redis:
condition: service_started
y-provider:
image: lasuite/impress-y-provider:latest
user: ${DOCKER_USER:-1000}
env_file:
- env.d/common
- env.d/yprovider
frontend:
image: lasuite/impress-frontend:latest
user: "101"
entrypoint:
- /docker-entrypoint.sh
command: ["nginx", "-g", "daemon off;"]
env_file:
- env.d/common
# Uncomment and set your values if using our nginx proxy example
#environment:
# - VIRTUAL_HOST=${DOCS_HOST} # used by nginx proxy
# - VIRTUAL_PORT=8083 # used by nginx proxy
# - LETSENCRYPT_HOST=${DOCS_HOST} # used by lets encrypt to generate TLS certificate
volumes:
- ./default.conf.template:/etc/nginx/templates/docs.conf.template
depends_on:
backend:
condition: service_healthy
# Uncomment if using our nginx proxy example
# networks:
# - proxy-tier
# - default
# Uncomment if using our nginx proxy example
#networks:
# proxy-tier:
# external: true

View File

@@ -0,0 +1,88 @@
# Deploy and Configure Keycloak for Docs
## Installation
> \[!CAUTION\]
> We provide those instructions as an example, for production environments, you should follow the [official documentation](https://www.keycloak.org/documentation).
### Step 1: Prepare your working environment:
```bash
mkdir keycloak
curl -o compose.yaml https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/docs/examples/compose/keycloak/compose.yaml
curl -o env.d/kc_postgresql https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/env.d/production.dist/kc_postgresql
curl -o env.d/keycloak https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/env.d/production.dist/keycloak
```
### Step 2:. Update `env.d/` files
The following variables need to be updated with your own values, others can be left as is:
```env
POSTGRES_PASSWORD=<generate postgres password>
KC_HOSTNAME=https://id.yourdomain.tld # Change with your own URL
KC_BOOTSTRAP_ADMIN_PASSWORD=<generate your password>
```
### Step 3: Expose keycloak instance on https
> \[!NOTE\]
> You can skip this section if you already have your own setup.
To access your Keycloak instance on the public network, it needs to be exposed on a domain with SSL termination. You can use our [example with nginx proxy and Let's Encrypt companion](../nginx-proxy/README.md) for automated creation/renewal of certificates using [acme.sh](http://acme.sh).
If following our example, uncomment the environment and network sections in compose file and update it with your values.
```yaml
version: '3'
services:
keycloak:
...
# Uncomment and set your values if using our nginx proxy example
# environment:
# - VIRTUAL_HOST=id.yourdomain.tld # used by nginx proxy
# - VIRTUAL_PORT=8080 # used by nginx proxy
# - LETSENCRYPT_HOST=id.yourdomain.tld # used by lets encrypt to generate TLS certificate
...
# Uncomment if using our nginx proxy example
# networks:
# - proxy-tier
# - default
# Uncomment if using our nginx proxy example
#networks:
# proxy-tier:
# external: true
```
### Step 4: Start the service
```bash
`docker compose up -d`
```
Your keycloak instance is now available on https://doc.yourdomain.tld
## Creating an OIDC Client for Docs Application
### Step 1: Create a New Realm
1. Log in to the Keycloak administration console.
2. Navigate to the realm tab and click on the "Create realm" button.
3. Enter the name of the realm - `docs`.
4. Click "Create".
#### Step 2: Create a New Client
1. Navigate to the "Clients" tab.
2. Click on the "Create client" button.
3. Enter the client ID - e.g. `docs`.
4. Enable "Client authentication" option.
6. Set the "Valid redirect URIs" to the URL of your docs application suffixed with `/*` - e.g., "https://docs.example.com/*".
1. Set the "Web Origins" to the URL of your docs application - e.g. `https://docs.example.com`.
1. Click "Save".
#### Step 3: Get Client Credentials
1. Go to the "Credentials" tab.
2. Copy the client ID (`docs` in this example) and the client secret.

View File

@@ -0,0 +1,36 @@
services:
kc_postgresql:
image: postgres:16
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 1s
timeout: 2s
retries: 300
env_file:
- env.d/kc_postgresql
volumes:
- ./data/keycloak:/var/lib/postgresql/data/pgdata
keycloak:
image: quay.io/keycloak/keycloak:26.1.3
command: ["start"]
env_file:
- env.d/kc_postgresql
- env.d/keycloak
# Uncomment and set your values if using our nginx proxy example
# environment:
# - VIRTUAL_HOST=id.yourdomain.tld # used by nginx proxy
# - VIRTUAL_PORT=8080 # used by nginx proxy
# - LETSENCRYPT_HOST=id.yourdomain.tld # used by lets encrypt to generate TLS certificate
depends_on:
kc_postgresql::
condition: service_healthy
restart: true
# Uncomment if using our nginx proxy example
# networks:
# - proxy-tier
# - default
#
#networks:
# proxy-tier:
# external: true

View File

@@ -0,0 +1,103 @@
# Deploy and Configure Minio for Docs
## Installation
> \[!CAUTION\]
> We provide those instructions as an example, it should not be run in production. For production environments, deploy MinIO [in a Multi-Node Multi-Drive (Distributed)](https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html#minio-mnmd) topology
### Step 1: Prepare your working environment:
```bash
mkdir minio
curl -o compose.yaml https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/docs/examples/compose/minio/compose.yaml
```
### Step 2:. Update compose file with your own values
```yaml
version: '3'
services:
minio:
...
environment:
- MINIO_ROOT_USER=<Set minio root username>
- MINIO_ROOT_PASSWORD=<Set minio root password>
```
### Step 3: Expose MinIO instance
#### Option 1: Internal network
You may not need to expose your MinIO instance to the public if only services hosted on the same private network need to access to your MinIO instance.
You should create a docker network that will be shared between those services
```bash
docker network create storage-tier
```
#### Option 2: Public network
If you want to expose your MinIO instance to the public, it needs to be exposed on a domain with SSL termination. You can use our [example](../nginx-proxy/README.md) with an nginx proxy and Let's Encrypt companion for automated creation/renewal of Let's Encrypt certificates using [acme.sh](http://acme.sh).
If following our example, uncomment the environment and network sections in compose file and update it with your values.
```yaml
version: '3'
services:
docs:
...
minio:
...
environment:
...
# - VIRTUAL_HOST=storage.yourdomain.tld # used by nginx proxy
# - VIRTUAL_PORT=9000 # used by nginx proxy
# - LETSENCRYPT_HOST=storage.yourdomain.tld # used by lets encrypt to generate TLS certificate
...
# Uncomment if using our nginx proxy example
# networks:
# - proxy-tier
# - default
# Uncomment if using our nginx proxy example
#networks:
# proxy-tier:
# external: true
```
In this example we are only exposing MinIO API service. Follow the official documentation to configure Minio WebUI.
### Step 4: Start the service
```bash
`docker compose up -d`
```
Your minio instance is now available on https://storage.yourdomain.tld
## Creating a user and bucket for your Docs instance
### Installing mc
Follow the [official documentation](https://min.io/docs/minio/linux/reference/minio-mc.html#install-mc) to install mc
### Step 1: Configure `mc` to connect to your MinIO Server with your root user
```shellscript
mc alias set minio <MINIO_SERVER_URL> <MINIO_ROOT_USER> <MINIO_ROOT_PASSWORD>
```
Replace the values with those you have set in the previous steps
### Step 2: Create a new bucket with versioning enabled
```shellscript
mc mb --with-versioning minio/<your-bucket-name>
```
Replace `your-bucket-name` with the desired name for your bucket e.g. `docs-media-storage`
### Additional notes:
For increased security you should create a dedicated user with `readwrite` access to the Bucket. In the following example we will use MinIO root user.

View File

@@ -0,0 +1,27 @@
services:
minio:
image: minio/minio
environment:
- MINIO_ROOT_USER=<set minio root username>
- MINIO_ROOT_PASSWORD=<set minio root password>
# Uncomment and set your values if using our nginx proxy example
# - VIRTUAL_HOST=storage.yourdomain.tld # used by nginx proxy
# - VIRTUAL_PORT=9000 # used by nginx proxy
# - LETSENCRYPT_HOST=storage.yourdomain.tld # used by lets encrypt to generate TLS certificate
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 1s
timeout: 20s
retries: 300
entrypoint: ""
command: minio server /data
volumes:
- ./data/minio:/data
# Uncomment if using our nginx proxy example
# networks:
# - proxy-tier
# Uncomment if using our nginx proxy example
#networks:
# proxy-tier:
# external: true

View File

@@ -0,0 +1,39 @@
# Nginx proxy with automatic SSL certificates
> \[!CAUTION\]
> We provide those instructions as an example, for extended development or production environments, you should follow the [official documentation](https://github.com/nginx-proxy/acme-companion/tree/main/docs).
Nginx-proxy sets up a container running nginx and docker-gen. docker-gen generates reverse proxy configs for nginx and reloads nginx when containers are started and stopped.
Acme-companion is a lightweight companion container for nginx-proxy. It handles the automated creation, renewal and use of SSL certificates for proxied Docker containers through the ACME protocol.
## Installation
### Step 1: Prepare your working environment:
```bash
mkdir nginx-proxy
curl -o compose.yaml https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/docs/examples/compose/nginx-proxy/compose.yaml
```
### Step 2: Edit `DEFAULT_EMAIL` in the compose file.
Albeit optional, it is recommended to provide a valid default email address through the `DEFAULT_EMAIL` environment variable, so that Let's Encrypt can warn you about expiring certificates and allow you to recover your account.
### Step 3: Create docker network
Containers need share the same network for auto-discovery.
```bash
docker network create proxy-tier
```
### Step 4: Start service
```bash
docker compose up -d
```
## Usage
Once both nginx-proxy and acme-companion containers are up and running, start any container you want proxied with environment variables `VIRTUAL_HOST` and `LETSENCRYPT_HOST` both set to the domain(s) your proxied container is going to use.

View File

@@ -0,0 +1,36 @@
services:
nginx-proxy:
image: nginxproxy/nginx-proxy
container_name: nginx-proxy
ports:
- "80:80"
- "443:443"
volumes:
- html:/usr/share/nginx/html
- certs:/etc/nginx/certs:ro
- /var/run/docker.sock:/tmp/docker.sock:ro
networks:
- proxy-tier
acme-companion:
image: nginxproxy/acme-companion
container_name: nginx-proxy-acme
environment:
- DEFAULT_EMAIL=mail@yourdomain.tld
volumes_from:
- nginx-proxy
volumes:
- certs:/etc/nginx/certs:rw
- acme:/etc/acme.sh
- /var/run/docker.sock:/var/run/docker.sock:ro
networks:
- proxy-tier
networks:
proxy-tier:
external: true
volumes:
html:
certs:
acme:

View File

@@ -33,8 +33,8 @@ backend:
OIDC_RP_SIGN_ALGO: RS256
OIDC_RP_SCOPES: "openid email"
OIDC_VERIFY_SSL: False
USER_OIDC_FIELD_TO_SHORTNAME: "given_name"
USER_OIDC_FIELDS_TO_FULLNAME: "given_name,usual_name"
OIDC_USERINFO_SHORTNAME_FIELD: "given_name"
OIDC_USERINFO_FULLNAME_FIELDS: "given_name,usual_name"
OIDC_REDIRECT_ALLOWED_HOSTS: https://impress.127.0.0.1.nip.io
OIDC_AUTH_REQUEST_EXTRA_PARAMS: "{'acr_values': 'eidas1'}"
LOGIN_REDIRECT_URL: https://impress.127.0.0.1.nip.io
@@ -82,13 +82,13 @@ backend:
python manage.py createsuperuser --email admin@example.com --password admin
restartPolicy: Never
# Exra volume to manage our local custom CA and avoid to set ssl_verify: false
# Extra volume to manage our local custom CA and avoid to set ssl_verify: false
extraVolumeMounts:
- name: certs
mountPath: /usr/local/lib/python3.12/site-packages/certifi/cacert.pem
mountPath: /usr/local/lib/python3.13/site-packages/certifi/cacert.pem
subPath: cacert.pem
# Exra volume to manage our local custom CA and avoid to set ssl_verify: false
# Extra volume to manage our local custom CA and avoid to set ssl_verify: false
extraVolumes:
- name: certs
configMap:
@@ -121,6 +121,22 @@ yProvider:
COLLABORATION_SERVER_ORIGIN: https://impress.127.0.0.1.nip.io
COLLABORATION_SERVER_SECRET: my-secret
Y_PROVIDER_API_KEY: my-secret
COLLABORATION_BACKEND_BASE_URL: https://impress.127.0.0.1.nip.io
NODE_EXTRA_CA_CERTS: /usr/local/share/ca-certificates/cacert.pem
# Mount the certificate so yProvider can establish tls with the backend
extraVolumeMounts:
- name: certs
mountPath: /usr/local/share/ca-certificates/cacert.pem
subPath: cacert.pem
extraVolumes:
- name: certs
configMap:
name: certifi
items:
- key: cacert.pem
path: cacert.pem
posthog:
ingress:
@@ -135,9 +151,6 @@ ingress:
ingressCollaborationWS:
enabled: true
host: impress.127.0.0.1.nip.io
annotations:
nginx.ingress.kubernetes.io/auth-url: https://impress.127.0.0.1.nip.io/api/v1.0/documents/collaboration-auth/
ingressCollaborationApi:
enabled: true

View File

@@ -91,7 +91,7 @@ extraDeploy:
},
{
"username": "user-e2e-chromium",
"email": "user@chromium.e2e",
"email": "user@chromium.test",
"firstName": "E2E",
"lastName": "Chromium",
"enabled": "true",
@@ -105,7 +105,7 @@ extraDeploy:
},
{
"username": "user-e2e-webkit",
"email": "user@webkit.e2e",
"email": "user@webkit.test",
"firstName": "E2E",
"lastName": "Webkit",
"enabled": "true",
@@ -119,7 +119,7 @@ extraDeploy:
},
{
"username": "user-e2e-firefox",
"email": "user@firefox.e2e",
"email": "user@firefox.test",
"firstName": "E2E",
"lastName": "Firefox",
"enabled": "true",

View File

@@ -0,0 +1,226 @@
# Installation with docker compose
We provide a sample configuration for running Docs using Docker Compose. Please note that this configuration is experimental, and the official way to deploy Docs in production is to use [k8s](../installation/k8s.md)
## Requirements
- A modern version of Docker and its Compose plugin.
- A domain name and DNS configured to your server.
- An Identity Provider that supports OpenID Connect protocol - we provide [an example to deploy Keycloak](../examples/compose/keycloak/README.md).
- An Object Storage that implements S3 API - we provide [an example to deploy Minio](../examples/compose/minio/README.md).
- A Postgresql database - we provide [an example in the compose file](../examples/compose/compose.yaml).
- A Redis database - we provide [an example in the compose file](../examples/compose/compose.yaml).
## Software Requirements
Ensure you have Docker Compose(v2) installed on your host server. Follow the official guidelines for a reliable setup:
Docker Compose is included with Docker Engine:
- **Docker Engine:** We suggest adhering to the instructions provided by Docker
for [installing Docker Engine](https://docs.docker.com/engine/install/).
For older versions of Docker Engine that do not include Docker Compose:
- **Docker Compose:** Install it as per the [official documentation](https://docs.docker.com/compose/install/).
> [!NOTE]
> `docker-compose` may not be supported. You are advised to use `docker compose` instead.
## Step 1: Prepare your working environment:
```bash
mkdir -p docs/env.d
curl -o compose.yaml https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/docs/examples/compose/compose.yaml
curl -o env.d/common https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/env.d/production.dist/common
curl -o env.d/backend https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/env.d/production.dist/backend
curl -o env.d/yprovider https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/env.d/production.dist/yprovider
curl -o env.d/common https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/env.d/production.dist/postgresql
```
## Step 2: Configuration
Docs configuration is achieved through environment variables. We provide a [detailed description of all variables](../env.md).
In this example, we assume the following services:
- OIDC provider on https://id.yourdomain.tld
- Object Storage on https://storage.yourdomain.tld
- Docs on https://docs.yourdomain.tld
- Bucket name is docs-media-storage
**Set your own values in `env.d/common`**
### OIDC
Authentication in Docs is managed through Open ID Connect protocol. A functional Identity Provider implementing this protocol is required.
For guidance, refer to our [Keycloak deployment example](../examples/compose/keycloak/README.md).
If using Keycloak as your Identity Provider, set `OIDC_RP_CLIENT_ID` and `OIDC_RP_CLIENT_SECRET` variables with those of the OIDC client created for Docs. By default we have set `docs` as the realm name, if you have named your realm differently, update the value `REALM_NAME` in `env.d/common`
For others OIDC providers, update the variables in `env.d/backend`.
### Object Storage
Files and media are stored in an Object Store that supports the S3 API.
For guidance, refer to our [Minio deployment example](../examples/compose/minio/README.md).
Set `AWS_S3_ACCESS_KEY_ID` and `AWS_S3_SECRET_ACCESS_KEY` with the credentials of a user with `readwrite` access to the bucket created for Docs.
### Postgresql
Docs uses PostgreSQL as its database. Although an external PostgreSQL can be used, our example provides a deployment method.
If you are using the example provided, you need to generate a secure key for `DB_PASSWORD` and set it in `env.d/postgresql`.
If you are using an external service or not using our default values, you should update the variables in `env.d/postgresql`
### Redis
Docs uses Redis for caching. While an external Redis can be used, our example provides a deployment method.
If you are using an external service, you need to set `REDIS_URL` environment variable in `env.d/backend`.
### Y Provider
The Y provider service enables collaboration through websockets.
Generates a secure key for `Y_PROVIDER_API_KEY` and `COLLABORATION_SERVER_SECRET` in ``env.d/yprovider``.
### Docs
The Docs backend is built on the Django Framework.
Generates a secure key for `DJANGO_SECRET_KEY` in `env.d/backend`.
### Logging
Update the following variables in `env.d/backend` if you want to change the logging levels:
```env
LOGGING_LEVEL_HANDLERS_CONSOLE=DEBUG
LOGGING_LEVEL_LOGGERS_ROOT=DEBUG
LOGGING_LEVEL_LOGGERS_APP=DEBUG
```
### Mail
The following environment variables are required in `env.d/backend` for the mail service to send invitations :
```env
DJANGO_EMAIL_HOST=<smtp host>
DJANGO_EMAIL_HOST_USER=<smtp user>
DJANGO_EMAIL_HOST_PASSWORD=<smtp password>
DJANGO_EMAIL_PORT=<smtp port>
DJANGO_EMAIL_FROM=<your email address>
#DJANGO_EMAIL_USE_TLS=true # A flag to enable or disable TLS for email sending.
#DJANGO_EMAIL_USE_SSL=true # A flag to enable or disable SSL for email sending.
DJANGO_EMAIL_BRAND_NAME=<brand name used in email templates> # e.g. "La Suite Numérique"
DJANGO_EMAIL_LOGO_IMG=<logo image to use in email templates.> # e.g. "https://docs.yourdomain.tld/assets/logo-suite-numerique.png"
```
### AI
Built-in AI actions let users generate, summarize, translate, and correct content.
AI is disabled by default. To enable it, the following environment variables must be set in in `env.d/backend`:
```env
AI_FEATURE_ENABLED=true # is false by default
AI_BASE_URL=https://openaiendpoint.com
AI_API_KEY=<API key>
AI_MODEL=<model used> e.g. llama
```
### Frontend theme
You can [customize your Docs instance](../theming.md) with your own theme and custom css.
The following environment variables must be set in `env.d/backend`:
```env
FRONTEND_THEME=default # name of your theme built with cuningham
FRONTEND_CSS_URL=https://storage.yourdomain.tld/themes/custom.css # custom css
```
## Step 3: Reverse proxy and SSL/TLS
> [!WARNING]
> In a production environment, configure SSL/TLS termination to run your instance on https.
If you have your own certificates and proxy setup, you can skip this part.
You can follow our [nginx proxy example](../examples/compose/nginx-proxy/README.md) with automatic generation and renewal of certificate with Let's Encrypt.
You will need to uncomment the environment and network sections in compose file and update it with your values.
```yaml
frontend:
...
# Uncomment and set your values if using our nginx proxy example
#environment:
# - VIRTUAL_HOST=${DOCS_HOST} # used by nginx proxy
# - VIRTUAL_PORT=8083 # used by nginx proxy
# - LETSENCRYPT_HOST=${DOCS_HOST} # used by lets encrypt to generate TLS certificate
...
# Uncomment if using our nginx proxy example
# networks:
# - proxy-tier
#
#networks:
# proxy-tier:
# external: true
```
## Step 4: Start Docs
You are ready to start your Docs application !
```bash
docker compose up -d
```
> [!NOTE]
> Version of the images are set to latest, you should pin it to the desired version to avoid unwanted upgrades when pulling latest image.
## Step 5: Run the database migration and create Django admin user
```bash
docker compose run --rm backend python manage.py migrate
docker compose run --rm backend python manage.py createsuperuser --email <admin email> --password <admin password>
```
Replace `<admin email>` with the email of your admin user and generate a secure password.
Your docs instance is now available on the domain you defined, https://docs.yourdomain.tld.
THe admin interface is available on https://docs.yourdomain.tld/admin with the admin user you just created.
## How to upgrade your Docs application
Before running an upgrade you must check the [Upgrade document](../../UPGRADE.md) for specific procedures that might be needed.
You can also check the [Changelog](../../CHANGELOG.md) for brief summary of the changes.
### Step 1: Edit the images tag with the desired version
### Step 2: Pull the images
```bash
docker compose pull
```
### Step 3: Restart your containers
```bash
docker compose restart
```
### Step 4: Run the database migration
Your database schema may need to be updated, run:
```bash
docker compose run --rm backend python manage.py migrate
```

View File

@@ -1,21 +1,20 @@
# Installation on a k8s cluster
This document is a step-by-step guide that describes how to install Docs on a k8s cluster without AI features. It's a teaching document to learn how it's work. It needs to be adapt for production environment.
This document is a step-by-step guide that describes how to install Docs on a k8s cluster without AI features. It's a teaching document to learn how it works. It needs to be adapted for a production environment.
## Prerequisites
- k8s cluster with an nginx-ingress controller
- an OIDC provider (if you don't have one, we will provide an example)
- a PostgreSQL server (if you don't have one, we will provide an example)
- a Memcached server (if you don't have one, we will provide an example)
- a S3 bucket (if you don't have one, we will provide an example)
- an OIDC provider (if you don't have one, we provide an example)
- a PostgreSQL server (if you don't have one, we provide an example)
- a Memcached server (if you don't have one, we provide an example)
- a S3 bucket (if you don't have one, we provide an example)
### Test cluster
If you do not have a test cluster, you can install everything on a local kind cluster. In this case, the simplest way is to use our script **bin/start-kind.sh**.
If you do not have a test cluster, you can install everything on a local Kind cluster. In this case, the simplest way is to use our script **bin/start-kind.sh**.
To be able to use the script, you will need to install:
To be able to use the script, you need to install:
- Docker (https://docs.docker.com/desktop/)
- Kind (https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
@@ -23,7 +22,7 @@ To be able to use the script, you will need to install:
- Helm (https://helm.sh/docs/intro/quickstart/#install-helm)
```
./bin/start-kind.sh
./bin/start-kind.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 4700 100 4700 0 0 92867 0 --:--:-- --:--:-- --:--:-- 94000
@@ -46,11 +45,11 @@ It will expire on 24 March 2027 🗓
2. Create kind cluster with containerd registry config dir enabled
Creating cluster "suite" ...
✓ Ensuring node image (kindest/node:v1.27.3) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-suite"
You can now use your cluster with:
@@ -96,13 +95,14 @@ ingress-nginx-admission-create-t55ph 0/1 Completed 0 2m56s
ingress-nginx-admission-patch-94dvt 0/1 Completed 1 2m56s
ingress-nginx-controller-57c548c4cd-2rx47 1/1 Running 0 2m56s
```
When your k8s cluster is ready (the ingress nginx controller is up), you can start the deployment. This cluster is special because it uses the *.127.0.0.1.nip.io domain and mkcert certificates to have full HTTPS support and easy domain name management.
Please remember that *.127.0.0.1.nip.io will always resolve to 127.0.0.1, except in the k8s cluster where we configure CoreDNS to answer with the ingress-nginx service IP.
When your k8s cluster is ready (the ingress nginx controller is up), you can start the deployment. This cluster is special because it uses the `*.127.0.0.1.nip.io` domain and mkcert certificates to have full HTTPS support and easy domain name management.
Please remember that `*.127.0.0.1.nip.io` will always resolve to `127.0.0.1`, except in the k8s cluster where we configure CoreDNS to answer with the ingress-nginx service IP.
## Preparation
### What will you use to authenticate your users ?
### What do you use to authenticate your users?
Docs uses OIDC, so if you already have an OIDC provider, obtain the necessary information to use it. In the next step, we will see how to configure Django (and thus Docs) to use it. If you do not have a provider, we will show you how to deploy a local Keycloak instance (this is not a production deployment, just a demo).
@@ -117,9 +117,9 @@ keycloak-0 1/1 Running 0 6m48s
keycloak-postgresql-0 1/1 Running 0 6m48s
```
From here the important informations you will need are :
From here the important information you will need are:
```
```yaml
OIDC_OP_JWKS_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/certs
OIDC_OP_AUTHORIZATION_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/token
@@ -133,9 +133,9 @@ OIDC_RP_SCOPES: "openid email"
You can find these values in **examples/keycloak.values.yaml**
### Find redis server connexion values
### Find redis server connection values
Impress need a redis so we will start by deploying a redis :
Docs needs a redis so we start by deploying one:
```
$ helm install redis oci://registry-1.docker.io/bitnamicharts/redis -f examples/redis.values.yaml
@@ -146,9 +146,9 @@ keycloak-postgresql-0 1/1 Running 0 26m
redis-master-0 1/1 Running 0 35s
```
### Find postgresql connexion values
### Find postgresql connection values
Impress uses a postgresql db as backend so if you have a provider, obtain the necessary information to use it. If you do not have, you can install a postgresql testing environment as follow:
Docs uses a postgresql database as backend, so if you have a provider, obtain the necessary information to use it. If you don't, you can install a postgresql testing environment as follow:
```
$ helm install postgresql oci://registry-1.docker.io/bitnamicharts/postgresql -f examples/postgresql.values.yaml
@@ -160,9 +160,9 @@ postgresql-0 1/1 Running 0 14m
redis-master-0 1/1 Running 0 42s
```
From here important informations you will need are :
From here the important information you will need are:
```
```yaml
DB_HOST: postgres-postgresql
DB_NAME: impress
DB_USER: dinum
@@ -173,9 +173,9 @@ POSTGRES_USER: dinum
POSTGRES_PASSWORD: pass
```
### Find s3 bucket connexion values
### Find s3 bucket connection values
Impress uses a s3 bucket to store documents so if you have a provider obtain the necessary information to use it. If you do not have, you can install a local minio testing environment as follow:
Docs uses an s3 bucket to store documents, so if you have a provider obtain the necessary information to use it. If you don't, you can install a local minio testing environment as follow:
```
$ helm install minio oci://registry-1.docker.io/bitnamicharts/minio -f examples/minio.values.yaml
@@ -191,7 +191,7 @@ redis-master-0 1/1 Running 0 10m
## Deployment
Now you are ready to deploy Impress without AI. AI requiered more dependancies (openai API). To deploy impress you need to provide all previous informations to the helm chart.
Now you are ready to deploy Docs without AI. AI requires more dependencies (OpenAI API). To deploy Docs you need to provide all previous information to the helm chart.
```
$ helm repo add impress https://suitenumerique.github.io/docs/
@@ -214,7 +214,7 @@ redis-master-0 1/1 Running 0 20m
## Test your deployment
In order to test your deployment you have to login to your instance. If you use exclusively our examples you can do :
In order to test your deployment you have to log into your instance. If you exclusively use our examples you can do:
```
$ kubectl get ingress
@@ -227,5 +227,4 @@ impress-docs-ws <none> impress.127.0.0.1.nip.io localhost
keycloak <none> keycloak.127.0.0.1.nip.io localhost 80 49m
```
You can use impress on https://impress.127.0.0.1.nip.io. The provisionning user in keycloak is impress/impress.
You can use Docs at https://impress.127.0.0.1.nip.io. The provisionning user in keycloak is impress/impress.

110
docs/system-requirements.md Normal file
View File

@@ -0,0 +1,110 @@
# La Suite Docs System & Requirements (2025-06)
## 1. Quick-Reference Matrix (single VM / laptop)
| Scenario | RAM | vCPU | SSD | Notes |
| ------------------------- | ----- | ---- | ------- | ------------------------- |
| **Solo dev** | 8 GB | 4 | 15 GB | Hot-reload + one IDE |
| **Team QA** | 16 GB | 6 | 30 GB | Runs integration tests |
| **Prod ≤ 100 live users** | 32 GB | 8 + | 50 GB + | Scale linearly above this |
Memory is the first bottleneck; CPU matters only when Celery or the Next.js build is saturated.
> **Note:** Memory consumption varies by operating system. Windows tends to be more memory-hungry than Linux, so consider adding 10-20% extra RAM when running on Windows compared to Linux-based systems.
## 2. Development Environment Memory Requirements
| Service | Typical use | Rationale / source |
| ------------------------ | ----------------------------- | --------------------------------------------------------------------------------------- |
| PostgreSQL | **1 2 GB** | `shared_buffers` starting point ≈ 25% RAM ([postgresql.org][1]) |
| Keycloak | **≈ 1.3 GB** | 70% of limit for heap + ~300 MB non-heap ([keycloak.org][2]) |
| Redis | **≤ 256 MB** | Empty instance ≈ 3 MB; budget 256 MB to allow small datasets ([stackoverflow.com][3]) |
| MinIO | **2 GB (dev) / 32 GB (prod)**| Pre-allocates 12 GiB; docs recommend 32 GB per host for ≤ 100 Ti storage ([min.io][4]) |
| Django API (+ Celery) | **0.8 1.5 GB** | Empirical in-house metrics |
| Next.js frontend | **0.5 1 GB** | Dev build chain |
| Y-Provider (y-websocket) | **< 200 MB** | Large 40 MB YDoc called “big” in community thread ([discuss.yjs.dev][5]) |
| Nginx | **< 100 MB** | Static reverse-proxy footprint |
[1]: https://www.postgresql.org/docs/9.1/runtime-config-resource.html "PostgreSQL: Documentation: 9.1: Resource Consumption"
[2]: https://www.keycloak.org/high-availability/concepts-memory-and-cpu-sizing "Concepts for sizing CPU and memory resources - Keycloak"
[3]: https://stackoverflow.com/questions/45233052/memory-footprint-for-redis-empty-instance "Memory footprint for Redis empty instance - Stack Overflow"
[4]: https://min.io/docs/minio/kubernetes/upstream/operations/checklists/hardware.html "Hardware Checklist — MinIO Object Storage for Kubernetes"
[5]: https://discuss.yjs.dev/t/understanding-memory-requirements-for-production-usage/198 "Understanding memory requirements for production usage - Yjs Community"
> **Rule of thumb:** add 2 GB for OS/overhead, then sum only the rows you actually run.
## 3. Production Environment Memory Requirements
Production deployments differ significantly from development environments. The table below shows typical memory usage for production services:
| Service | Typical use | Rationale / notes |
| ------------------------ | ----------------------------- | --------------------------------------------------------------------------------------- |
| PostgreSQL | **2 8 GB** | Higher `shared_buffers` and connection pooling for concurrent users |
| OIDC Provider (optional) | **Variable** | Any OIDC-compatible provider (Keycloak, Auth0, Azure AD, etc.) - external or self-hosted |
| Redis | **256 MB 2 GB** | Session storage and caching; scales with active user sessions |
| Object Storage (optional)| **External or self-hosted** | Can use AWS S3, Azure Blob, Google Cloud Storage, or self-hosted MinIO |
| Django API (+ Celery) | **1 3 GB** | Production workloads with background tasks and higher concurrency |
| Static Files (Nginx) | **< 200 MB** | Serves Next.js build output and static assets; no development overhead |
| Y-Provider (y-websocket) | **200 MB 1 GB** | Scales with concurrent document editing sessions |
| Nginx (Load Balancer) | **< 200 MB** | Reverse proxy, SSL termination, static file serving |
### Production Architecture Notes
- **Frontend**: Uses pre-built Next.js static assets served by Nginx (no Node.js runtime needed)
- **Authentication**: Any OIDC-compatible provider can be used instead of self-hosted Keycloak
- **Object Storage**: External services (S3, Azure Blob) or self-hosted solutions (MinIO) are both viable
- **Database**: Consider PostgreSQL clustering or managed database services for high availability
- **Scaling**: Horizontal scaling is recommended for Django API and Y-Provider services
### Minimal Production Setup (Core Services Only)
| Service | Memory | Notes |
| ------------------------ | --------- | --------------------------------------- |
| PostgreSQL | **2 GB** | Core database |
| Django API (+ Celery) | **1.5 GB**| Backend services |
| Y-Provider | **200 MB**| Real-time collaboration |
| Nginx | **100 MB**| Static files + reverse proxy |
| Redis | **256 MB**| Session storage |
| **Total (without auth/storage)** | **≈ 4 GB** | External OIDC + object storage assumed |
## 4. Recommended Software Versions
| Tool | Minimum |
| ----------------------- | ------- |
| Docker Engine / Desktop | 24.0 |
| Docker Compose | v2 |
| Git | 2.40 |
| **Node.js** | 22+ |
| **Python** | 3.13+ |
| GNU Make | 4.4 |
| Kind | 0.22 |
| Helm | 3.14 |
| kubectl | 1.29 |
| mkcert | 1.4 |
## 5. Ports (dev defaults)
| Port | Service |
| --------- | --------------------- |
| 3000 | Next.js |
| 8071 | Django |
| 4444 | Y-Provider |
| 8080 | Keycloak |
| 8083 | Nginx proxy |
| 9000/9001 | MinIO |
| 15432 | PostgreSQL (main) |
| 5433 | PostgreSQL (Keycloak) |
| 1081 | MailCatcher |
## 6. Sizing Guidelines
**RAM** start at 8 GB dev / 16 GB staging / 32 GB prod. Postgres and Keycloak are the first to OOM; scale them first.
> **OS considerations:** Windows systems typically require 10-20% more RAM than Linux due to higher OS overhead. Docker Desktop on Windows also uses additional memory compared to native Linux Docker.
**CPU** budget one vCPU per busy container until Celery or Next.js builds saturate.
**Disk** SSD; add 10 GB extra for the Docker layer cache.
**MinIO** for demos, mount a local folder instead of running MinIO to save 2 GB+ of RAM.

70
docs/theming.md Normal file
View File

@@ -0,0 +1,70 @@
# Runtime Theming 🎨
### How to Use
To use this feature, simply set the `FRONTEND_CSS_URL` environment variable to the URL of your custom CSS file. For example:
```javascript
FRONTEND_CSS_URL=http://anything/custom-style.css
```
Once you've set this variable, our application will load your custom CSS file and apply the styles to our frontend application.
### Benefits
This feature provides several benefits, including:
* **Easy customization** 🔄: With this feature, you can easily customize the look and feel of our application without requiring any code changes.
* **Flexibility** 🌈: You can use any CSS styles you like to create a custom theme that meets your needs.
* **Runtime theming** ⏱️: This feature allows you to change the theme of our application at runtime, without requiring a restart or recompilation.
### Example Use Case
Let's say you want to change the background color of our application to a custom color. You can create a custom CSS file with the following contents:
```css
body {
background-color: #3498db;
}
```
Then, set the `FRONTEND_CSS_URL` environment variable to the URL of your custom CSS file. Once you've done this, our application will load your custom CSS file and apply the styles, changing the background color to the custom color you specified.
----
# **Footer Configuration** 📝
The footer is configurable from the theme customization file.
### Settings 🔧
```shellscript
THEME_CUSTOMIZATION_FILE_PATH=<path>
```
### Example of JSON
The json must follow some rules: https://github.com/suitenumerique/docs/blob/main/src/helm/env.d/dev/configuration/theme/demo.json
`footer.default` is the fallback if the language is not supported.
---
Below is a visual example of a configured footer ⬇️:
![Footer Configuration Example](./assets/footer-configurable.png)
----
# **Custom Translations** 📝
The translations can be partially overridden from the theme customization file.
### Settings 🔧
```shellscript
THEME_CUSTOMIZATION_FILE_PATH=<path>
```
### Example of JSON
The json must follow some rules: https://github.com/suitenumerique/docs/blob/main/src/helm/env.d/dev/configuration/theme/demo.json

194
docs/troubleshoot.md Normal file
View File

@@ -0,0 +1,194 @@
# Troubleshooting Guide
## Line Ending Issues on Windows (LF/CRLF)
### Problem Description
This project uses **LF (Line Feed: `\n`) line endings** exclusively. Windows users may encounter issues because:
- **Windows** defaults to CRLF (Carriage Return + Line Feed: `\r\n`) for line endings
- **This project** uses LF line endings for consistency across all platforms
- **Git** may automatically convert line endings, causing conflicts or build failures
### Common Symptoms
- Git shows files as modified even when no changes were made
- Error messages like "warning: LF will be replaced by CRLF"
- Build failures or linting errors due to line ending mismatches
### Solutions for Windows Users
#### Configure Git to Preserve LF (Recommended)
Configure Git to NOT convert line endings and preserve LF:
```bash
git config core.autocrlf false
git config core.eol lf
```
This tells Git to:
- Never convert line endings automatically
- Always use LF for line endings in working directory
#### Fix Existing Repository with Wrong Line Endings
If you already have CRLF line endings in your local repository, the **best approach** is to configure Git properly and clone the project again:
1. **Configure Git first**:
```bash
git config --global core.autocrlf false
git config --global core.eol lf
```
2. **Clone the project fresh** (recommended):
```bash
# Navigate to parent directory
cd ..
# Remove current repository (backup your changes first!)
rm -rf docs
# Clone again with correct line endings
git clone git@github.com:suitenumerique/docs.git
```
**Alternative**: If you have uncommitted changes and cannot re-clone:
1. **Backup your changes**:
```bash
git add .
git commit -m "Save changes before fixing line endings"
```
2. **Remove all files from Git's index**:
```bash
git rm --cached -r .
```
3. **Reset Git configuration** (if not done globally):
```bash
git config core.autocrlf false
git config core.eol lf
```
4. **Re-add all files** (Git will use LF line endings):
```bash
git add .
```
5. **Commit the changes**:
```bash
git commit -m "✏️(project) Fix line endings to LF"
```
## Minio Permission Issues on Windows
### Problem Description
On Windows, you may encounter permission-related errors when running Minio in development mode with Docker Compose. This typically happens because:
- **Windows file permissions** don't map well to Unix-style user IDs used in Docker containers
- **Docker Desktop** may have issues with user mapping when using the `DOCKER_USER` environment variable
- **Minio container** fails to start or access volumes due to permission conflicts
### Common Symptoms
- Minio container fails to start with permission denied errors
- Error messages related to file system permissions in Minio logs
- Unable to create or access buckets in the development environment
- Docker Compose showing Minio service as unhealthy or exited
### Solution for Windows Users
If you encounter Minio permission issues on Windows, you can temporarily disable user mapping for the Minio service:
1. **Open the `compose.yml` file**
2. **Comment out the user directive** in the `minio` service section:
```yaml
minio:
# user: ${DOCKER_USER:-1000} # Comment this line on Windows if permission issues occur
image: minio/minio
environment:
- MINIO_ROOT_USER=impress
- MINIO_ROOT_PASSWORD=password
# ... rest of the configuration
```
3. **Restart the services**:
```bash
make run
```
### Why This Works
- Commenting out the `user` directive allows the Minio container to run with its default user
- This bypasses Windows-specific permission mapping issues
- The container will have the necessary permissions to access and manage the mounted volumes
### Note
This is a **development-only workaround**. In production environments, proper user mapping and security considerations should be maintained according to your deployment requirements.
## Frontend File Watching Issues on Windows
### Problem Description
Windows users may experience issues with file watching in the frontend-development container. This typically happens because:
- **Docker on Windows** has known limitations with file change detection
- **Node.js file watchers** may not detect changes properly on Windows filesystem
- **Hot reloading** fails to trigger when files are modified
### Common Symptoms
- Changes to frontend code aren't detected automatically
- Hot module replacement doesn't work as expected
- Need to manually restart the frontend container after code changes
- Console shows no reaction when saving files
### Solution: Enable WATCHPACK_POLLING
Add the `WATCHPACK_POLLING=true` environment variable to the frontend-development service in your local environment:
1. **Modify the `compose.yml` file** by adding the environment variable to the frontend-development service:
```yaml
frontend-development:
user: "${DOCKER_USER:-1000}"
build:
context: .
dockerfile: ./src/frontend/Dockerfile
target: impress-dev
args:
API_ORIGIN: "http://localhost:8071"
PUBLISH_AS_MIT: "false"
SW_DEACTIVATED: "true"
image: impress:frontend-development
environment:
- WATCHPACK_POLLING=true # Add this line for Windows users
volumes:
- ./src/frontend:/home/frontend
- /home/frontend/node_modules
- /home/frontend/apps/impress/node_modules
ports:
- "3000:3000"
```
2. **Restart your containers**:
```bash
make run
```
### Why This Works
- `WATCHPACK_POLLING=true` forces the file watcher to use polling instead of filesystem events
- Polling periodically checks for file changes rather than relying on OS-level file events
- This is more reliable on Windows but slightly increases CPU usage
- Changes to your frontend code should now be detected properly, enabling hot reloading
### Note
This setting is primarily needed for Windows users. Linux and macOS users typically don't need this setting as file watching works correctly by default on those platforms.

View File

@@ -50,15 +50,18 @@ OIDC_REDIRECT_ALLOWED_HOSTS=["http://localhost:8083", "http://localhost:3000"]
OIDC_AUTH_REQUEST_EXTRA_PARAMS={"acr_values": "eidas1"}
# AI
AI_FEATURE_ENABLED=true
AI_BASE_URL=https://openaiendpoint.com
AI_API_KEY=password
AI_MODEL=llama
# Collaboration
COLLABORATION_API_URL=http://nginx:8083/collaboration/api/
COLLABORATION_API_URL=http://y-provider-development:4444/collaboration/api/
COLLABORATION_BACKEND_BASE_URL=http://app-dev:8000
COLLABORATION_SERVER_ORIGIN=http://localhost:3000
COLLABORATION_SERVER_SECRET=my-secret
COLLABORATION_WS_URL=ws://localhost:8083/collaboration/ws/
COLLABORATION_WS_URL=ws://localhost:4444/collaboration/ws/
# Frontend
FRONTEND_THEME=dsfr
DJANGO_SERVER_TO_SERVER_API_TOKENS=server-api-token
Y_PROVIDER_API_BASE_URL=http://y-provider-development:4444/api/
Y_PROVIDER_API_KEY=yprovider-api-key

View File

@@ -1,6 +1,9 @@
# For the CI job test-e2e
SUSTAINED_THROTTLE_RATES="200/hour"
BURST_THROTTLE_RATES="200/minute"
DJANGO_SERVER_TO_SERVER_API_TOKENS=test-e2e
Y_PROVIDER_API_KEY=yprovider-api-key
COLLABORATION_API_URL=http://y-provider:4444/collaboration/api/
SUSTAINED_THROTTLE_RATES="200/hour"
Y_PROVIDER_API_BASE_URL=http://y-provider:4444/api/
THEME_CUSTOMIZATION_FILE_PATH="" #force theme_customization to be empty
#COLLABORATION_API_URL=http://y-provider:4444/collaboration/api/
#Y_PROVIDER_API_BASE_URL=http://y-provider:4444/api/

View File

@@ -0,0 +1,65 @@
## Django
DJANGO_ALLOWED_HOSTS=${DOCS_HOST}
DJANGO_SECRET_KEY=<generate a random key>
DJANGO_SETTINGS_MODULE=impress.settings
DJANGO_CONFIGURATION=Production
# Logging
# Set to DEBUG level for dev only
LOGGING_LEVEL_HANDLERS_CONSOLE=ERROR
LOGGING_LEVEL_LOGGERS_ROOT=INFO
LOGGING_LEVEL_LOGGERS_APP=INFO
# Python
PYTHONPATH=/app
# Mail
DJANGO_EMAIL_HOST=<smtp host>
DJANGO_EMAIL_HOST_USER=<smtp user>
DJANGO_EMAIL_HOST_PASSWORD=<smtp password>
DJANGO_EMAIL_PORT=<smtp port>
DJANGO_EMAIL_FROM=<your email address>
#DJANGO_EMAIL_USE_TLS=true # A flag to enable or disable TLS for email sending.
#DJANGO_EMAIL_USE_SSL=true # A flag to enable or disable SSL for email sending.
DJANGO_EMAIL_BRAND_NAME="La Suite Numérique"
DJANGO_EMAIL_LOGO_IMG="https://${DOCS_HOST}/assets/logo-suite-numerique.png"
# Media
AWS_S3_ENDPOINT_URL=https://${S3_HOST}
AWS_S3_ACCESS_KEY_ID=<s3 access key>
AWS_S3_SECRET_ACCESS_KEY=<s3 secret key>
AWS_STORAGE_BUCKET_NAME=${BUCKET_NAME}
MEDIA_BASE_URL=https://${DOCS_HOST}
# OIDC
OIDC_OP_JWKS_ENDPOINT=https://${KEYCLOAK_HOST}/realms/${REALM_NAME}/protocol/openid-connect/certs
OIDC_OP_AUTHORIZATION_ENDPOINT=https://${KEYCLOAK_HOST}/realms/${REALM_NAME}/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT=https://${KEYCLOAK_HOST}/realms/${REALM_NAME}/protocol/openid-connect/token
OIDC_OP_USER_ENDPOINT=https://${KEYCLOAK_HOST}/realms/${REALM_NAME}/protocol/openid-connect/userinfo
OIDC_OP_LOGOUT_ENDPOINT=https://${KEYCLOAK_HOST}/realms/${REALM_NAME}/protocol/openid-connect/logout
OIDC_RP_CLIENT_ID=<client_id>
OIDC_RP_CLIENT_SECRET=<client secret>
OIDC_RP_SIGN_ALGO=RS256
OIDC_RP_SCOPES="openid email"
#USER_OIDC_FIELD_TO_SHORTNAME
#USER_OIDC_FIELDS_TO_FULLNAME
LOGIN_REDIRECT_URL=https://${DOCS_HOST}
LOGIN_REDIRECT_URL_FAILURE=https://${DOCS_HOST}
LOGOUT_REDIRECT_URL=https://${DOCS_HOST}
OIDC_REDIRECT_ALLOWED_HOSTS=["https://${DOCS_HOST}"]
# AI
#AI_FEATURE_ENABLED=true # is false by default
#AI_BASE_URL=https://openaiendpoint.com
#AI_API_KEY=<API key>
#AI_MODEL=<model used> e.g. llama
# Frontend
#FRONTEND_THEME=mytheme
#FRONTEND_CSS_URL=https://storage.yourdomain.tld/themes/custom.css
#FRONTEND_FOOTER_FEATURE_ENABLED=true
#FRONTEND_URL_JSON_FOOTER=https://docs.domain.tld/contents/footer-demo.json

View File

@@ -0,0 +1,9 @@
DOCS_HOST=docs.domain.tld
KEYCLOAK_HOST=id.domain.tld
S3_HOST=storage.domain.tld
BACKEND_HOST=backend
FRONTEND_HOST=frontend
YPROVIDER_HOST=y-provider
BUCKET_NAME=docs-media-storage
REALM_NAME=docs
#COLLABORATION_WS_URL=wss://${DOCS_HOST}/collaboration/ws/

View File

@@ -0,0 +1,13 @@
# Postgresql db container configuration
POSTGRES_DB=keycloak
POSTGRES_USER=keycloak
POSTGRES_PASSWORD=<generate postgres password>
PGDATA=/var/lib/postgresql/data/pgdata
# Keycloak postgresql configuration
KC_DB=postgres
KC_DB_SCHEMA=public
KC_DB_HOST=postgresql
KC_DB_NAME=${POSTGRES_DB}
KC_DB_USER=${POSTGRES_USER}
KC_DB_PASSWORD=${POSTGRES_PASSWORD}

View File

@@ -0,0 +1,8 @@
# Keycloak admin user
KC_BOOTSTRAP_ADMIN_USERNAME=admin
KC_BOOTSTRAP_ADMIN_PASSWORD=<generate your password>
# Keycloak configuration
KC_HOSTNAME=https://id.yourdomain.tld # Change with your own URL
KC_PROXY_HEADERS=xforwarded # in this example we are running behind an nginx proxy
KC_HTTP_ENABLED=true # in this example we are running behind an nginx proxy

View File

@@ -0,0 +1,11 @@
# App database configuration
DB_HOST=postgresql
DB_NAME=docs
DB_USER=docs
DB_PASSWORD=<generate a secure password>
DB_PORT=5432
# Postgresql db container configuration
POSTGRES_DB=docs
POSTGRES_USER=docs
POSTGRES_PASSWORD=${DB_PASSWORD}

View File

@@ -0,0 +1,7 @@
Y_PROVIDER_API_BASE_URL=http://${YPROVIDER_HOST}:4444/api
Y_PROVIDER_API_KEY=<generate a random key>
COLLABORATION_SERVER_SECRET=<generate a random key>
COLLABORATION_SERVER_ORIGIN=https://${DOCS_HOST}
COLLABORATION_API_URL=https://${DOCS_HOST}/collaboration/api/
COLLABORATION_BACKEND_BASE_URL=https://${DOCS_HOST}
COLLABORATION_LOGGING=true

View File

@@ -31,7 +31,7 @@ class GitmojiTitle(LineRule):
"https://raw.githubusercontent.com/carloscuesta/gitmoji/master/packages/gitmojis/src/gitmojis.json"
).json()["gitmojis"]
emojis = [item["emoji"] for item in gitmojis]
pattern = r"^({:s})\(.*\)\s[a-z].*$".format("|".join(emojis))
pattern = r"^({:s})\(.*\)\s[a-zA-Z].*$".format("|".join(emojis))
if not re.search(pattern, title):
violation_msg = 'Title does not match regex "<gitmoji>(<scope>) <subject>"'
return [RuleViolation(self.id, violation_msg, title)]

27
publiccode.yml Normal file
View File

@@ -0,0 +1,27 @@
publiccodeYmlVersion: "2.4.0"
name: Docs
url: https://github.com/suitenumerique/docs
landingURL: https://github.com/suitenumerique/docs
creationDate: 2023-12-10
logo: https://raw.githubusercontent.com/suitenumerique/docs/main/docs/assets/docs-logo.png
usedBy:
- Direction interministériel du numérique (DINUM)
fundedBy:
- name: Direction interministériel du numérique (DINUM)
url: https://www.numerique.gouv.fr
roadmap: "https://github.com/orgs/suitenumerique/projects/2/views/1"
softwareType: "standalone/other"
description:
en:
shortDescription: "The open source document editor where your notes can become knowledge through live collaboration"
fr:
shortDescription: "L'éditeur de documents open source où vos notes peuvent devenir des connaissances grâce à la collaboration en direct."
legal:
license: MIT
maintenance:
type: internal
contacts:
- name: "Virgile Deville"
email: "virgile.deville@numerique.gouv.fr"
- name: "samuel.paccoud"
email: "samuel.paccoud@numerique.gouv.fr"

View File

@@ -1,7 +1,7 @@
{
"extends": ["github>numerique-gouv/renovate-configuration"],
"dependencyDashboard": true,
"labels": ["dependencies", "noChangeLog"],
"labels": ["dependencies", "noChangeLog", "automated"],
"packageRules": [
{
"enabled": false,
@@ -9,20 +9,24 @@
"matchManagers": ["pep621"],
"matchPackageNames": []
},
{
"groupName": "allowed redis versions",
"matchManagers": ["pep621"],
"matchPackageNames": ["redis"],
"allowedVersions": "<6.0.0"
},
{
"enabled": false,
"groupName": "ignored js dependencies",
"matchManagers": ["npm"],
"matchPackageNames": [
"@openfun/cunningham-react",
"@types/react",
"@types/react-dom",
"@hocuspocus/provider",
"@hocuspocus/server",
"docx",
"eslint",
"fetch-mock",
"node",
"node-fetch",
"react",
"react-dom",
"workbox-webpack-plugin"
]
}

0
secu-audit.md Normal file
View File

View File

@@ -151,6 +151,8 @@ class DocumentAdmin(TreeAdmin):
"path",
"depth",
"numchild",
"duplicated_from",
"attachments",
)
},
),
@@ -166,8 +168,10 @@ class DocumentAdmin(TreeAdmin):
"updated_at",
)
readonly_fields = (
"attachments",
"creator",
"depth",
"duplicated_from",
"id",
"numchild",
"path",

View File

@@ -17,9 +17,10 @@ def exception_handler(exc, context):
https://gist.github.com/twidi/9d55486c36b6a51bdcb05ce3a763e79f
"""
if isinstance(exc, ValidationError):
detail = exc.message_dict
if hasattr(exc, "message"):
detail = None
if hasattr(exc, "message_dict"):
detail = exc.message_dict
elif hasattr(exc, "message"):
detail = exc.message
elif hasattr(exc, "messages"):
detail = exc.messages

View File

@@ -1,5 +1,7 @@
"""API filters for Impress' core application."""
import unicodedata
from django.utils.translation import gettext_lazy as _
import django_filters
@@ -7,7 +9,50 @@ import django_filters
from core import models
def remove_accents(value):
"""Remove accents from a string (vélo -> velo)."""
return "".join(
c
for c in unicodedata.normalize("NFD", value)
if unicodedata.category(c) != "Mn"
)
class AccentInsensitiveCharFilter(django_filters.CharFilter):
"""
A custom CharFilter that filters on the accent-insensitive value searched.
"""
def filter(self, qs, value):
"""
Apply the filter to the queryset using the unaccented version of the field.
Args:
qs: The queryset to filter.
value: The value to search for in the unaccented field.
Returns:
A filtered queryset.
"""
if value:
value = remove_accents(value)
return super().filter(qs, value)
class DocumentFilter(django_filters.FilterSet):
"""
Custom filter for filtering documents on title (accent and case insensitive).
"""
title = AccentInsensitiveCharFilter(
field_name="title", lookup_expr="unaccent__icontains", label=_("Title")
)
class Meta:
model = models.Document
fields = ["title"]
class ListDocumentFilter(DocumentFilter):
"""
Custom filter for filtering documents.
"""
@@ -18,9 +63,6 @@ class DocumentFilter(django_filters.FilterSet):
is_favorite = django_filters.BooleanFilter(
method="filter_is_favorite", label=_("Favorite")
)
title = django_filters.CharFilter(
field_name="title", lookup_expr="icontains", label=_("Title")
)
class Meta:
model = models.Document

View File

@@ -6,6 +6,7 @@ from django.http import Http404
from rest_framework import permissions
from core import choices
from core.models import DocumentAccess, RoleChoices, get_trashbin_cutoff
ACTION_FOR_METHOD_TO_PERMISSION = {
@@ -96,26 +97,27 @@ class CanCreateInvitationPermission(permissions.BasePermission):
).exists()
class AccessPermission(permissions.BasePermission):
"""Permission class for access objects."""
class ResourceWithAccessPermission(permissions.BasePermission):
"""A permission class for templates and invitations."""
def has_permission(self, request, view):
"""check create permission for templates."""
return request.user.is_authenticated or view.action != "create"
def has_object_permission(self, request, view, obj):
"""Check permission for a given object."""
abilities = obj.get_abilities(request.user)
action = view.action
try:
action = ACTION_FOR_METHOD_TO_PERMISSION[view.action][request.method]
except KeyError:
pass
return abilities.get(action, False)
class DocumentAccessPermission(AccessPermission):
class DocumentPermission(permissions.BasePermission):
"""Subclass to handle soft deletion specificities."""
def has_permission(self, request, view):
"""check create permission for documents."""
return request.user.is_authenticated or view.action != "create"
def has_object_permission(self, request, view, obj):
"""
Return a 404 on deleted documents
@@ -127,10 +129,45 @@ class DocumentAccessPermission(AccessPermission):
) and deleted_at < get_trashbin_cutoff():
raise Http404
# Compute permission first to ensure the "user_roles" attribute is set
has_permission = super().has_object_permission(request, view, obj)
abilities = obj.get_abilities(request.user)
action = view.action
try:
action = ACTION_FOR_METHOD_TO_PERMISSION[view.action][request.method]
except KeyError:
pass
has_permission = abilities.get(action, False)
if obj.ancestors_deleted_at and not RoleChoices.OWNER in obj.user_roles:
raise Http404
return has_permission
class ResourceAccessPermission(IsAuthenticated):
"""Permission class for document access objects."""
def has_permission(self, request, view):
"""check create permission for accesses in documents tree."""
if super().has_permission(request, view) is False:
return False
if view.action == "create":
role = getattr(view, view.resource_field_name).get_role(request.user)
if role not in choices.PRIVILEGED_ROLES:
raise exceptions.PermissionDenied(
"You are not allowed to manage accesses for this resource."
)
return True
def has_object_permission(self, request, view, obj):
"""Check permission for a given object."""
abilities = obj.get_abilities(request.user)
requested_role = request.data.get("role")
if requested_role and requested_role not in abilities.get("set_role_to", []):
return False
action = view.action
return abilities.get(action, False)

View File

@@ -1,6 +1,8 @@
"""Client serializers for the impress core app."""
import binascii
import mimetypes
from base64 import b64decode
from django.conf import settings
from django.db.models import Q
@@ -8,9 +10,9 @@ from django.utils.functional import lazy
from django.utils.translation import gettext_lazy as _
import magic
from rest_framework import exceptions, serializers
from rest_framework import serializers
from core import enums, models
from core import choices, enums, models, utils
from core.services.ai_services import AI_ACTIONS
from core.services.converter_services import (
ConversionError,
@@ -23,137 +25,61 @@ class UserSerializer(serializers.ModelSerializer):
class Meta:
model = models.User
fields = ["id", "email", "full_name", "short_name"]
fields = ["id", "email", "full_name", "short_name", "language"]
read_only_fields = ["id", "email", "full_name", "short_name"]
class BaseAccessSerializer(serializers.ModelSerializer):
class UserLightSerializer(UserSerializer):
"""Serialize users with limited fields."""
class Meta:
model = models.User
fields = ["full_name", "short_name"]
read_only_fields = ["full_name", "short_name"]
class TemplateAccessSerializer(serializers.ModelSerializer):
"""Serialize template accesses."""
abilities = serializers.SerializerMethodField(read_only=True)
def update(self, instance, validated_data):
"""Make "user" field is readonly but only on update."""
validated_data.pop("user", None)
return super().update(instance, validated_data)
def get_abilities(self, access) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return access.get_abilities(request.user)
return {}
def validate(self, attrs):
"""
Check access rights specific to writing (create/update)
"""
request = self.context.get("request")
user = getattr(request, "user", None)
role = attrs.get("role")
# Update
if self.instance:
can_set_role_to = self.instance.get_abilities(user)["set_role_to"]
if role and role not in can_set_role_to:
message = (
f"You are only allowed to set role to {', '.join(can_set_role_to)}"
if can_set_role_to
else "You are not allowed to set this role for this template."
)
raise exceptions.PermissionDenied(message)
# Create
else:
try:
resource_id = self.context["resource_id"]
except KeyError as exc:
raise exceptions.ValidationError(
"You must set a resource ID in kwargs to create a new access."
) from exc
if not self.Meta.model.objects.filter( # pylint: disable=no-member
Q(user=user) | Q(team__in=user.teams),
role__in=[models.RoleChoices.OWNER, models.RoleChoices.ADMIN],
**{self.Meta.resource_field_name: resource_id}, # pylint: disable=no-member
).exists():
raise exceptions.PermissionDenied(
"You are not allowed to manage accesses for this resource."
)
if (
role == models.RoleChoices.OWNER
and not self.Meta.model.objects.filter( # pylint: disable=no-member
Q(user=user) | Q(team__in=user.teams),
role=models.RoleChoices.OWNER,
**{self.Meta.resource_field_name: resource_id}, # pylint: disable=no-member
).exists()
):
raise exceptions.PermissionDenied(
"Only owners of a resource can assign other users as owners."
)
# pylint: disable=no-member
attrs[f"{self.Meta.resource_field_name}_id"] = self.context["resource_id"]
return attrs
class DocumentAccessSerializer(BaseAccessSerializer):
"""Serialize document accesses."""
user_id = serializers.PrimaryKeyRelatedField(
queryset=models.User.objects.all(),
write_only=True,
source="user",
required=False,
allow_null=True,
)
user = UserSerializer(read_only=True)
class Meta:
model = models.DocumentAccess
resource_field_name = "document"
fields = ["id", "user", "user_id", "team", "role", "abilities"]
read_only_fields = ["id", "abilities"]
class TemplateAccessSerializer(BaseAccessSerializer):
"""Serialize template accesses."""
class Meta:
model = models.TemplateAccess
resource_field_name = "template"
fields = ["id", "user", "team", "role", "abilities"]
read_only_fields = ["id", "abilities"]
class BaseResourceSerializer(serializers.ModelSerializer):
"""Serialize documents."""
abilities = serializers.SerializerMethodField(read_only=True)
accesses = TemplateAccessSerializer(many=True, read_only=True)
def get_abilities(self, document) -> dict:
def get_abilities(self, instance) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return document.get_abilities(request.user)
return instance.get_abilities(request.user)
return {}
def update(self, instance, validated_data):
"""Make "user" field is readonly but only on update."""
validated_data.pop("user", None)
return super().update(instance, validated_data)
class ListDocumentSerializer(BaseResourceSerializer):
class ListDocumentSerializer(serializers.ModelSerializer):
"""Serialize documents with limited fields for display in lists."""
is_favorite = serializers.BooleanField(read_only=True)
nb_accesses = serializers.IntegerField(read_only=True)
user_roles = serializers.SerializerMethodField(read_only=True)
nb_accesses_ancestors = serializers.IntegerField(read_only=True)
nb_accesses_direct = serializers.IntegerField(read_only=True)
user_role = serializers.SerializerMethodField(read_only=True)
abilities = serializers.SerializerMethodField(read_only=True)
class Meta:
model = models.Document
fields = [
"id",
"abilities",
"ancestors_link_reach",
"ancestors_link_role",
"computed_link_reach",
"computed_link_role",
"created_at",
"creator",
"depth",
@@ -161,16 +87,21 @@ class ListDocumentSerializer(BaseResourceSerializer):
"is_favorite",
"link_role",
"link_reach",
"nb_accesses",
"nb_accesses_ancestors",
"nb_accesses_direct",
"numchild",
"path",
"title",
"updated_at",
"user_roles",
"user_role",
]
read_only_fields = [
"id",
"abilities",
"ancestors_link_reach",
"ancestors_link_role",
"computed_link_reach",
"computed_link_role",
"created_at",
"creator",
"depth",
@@ -178,22 +109,50 @@ class ListDocumentSerializer(BaseResourceSerializer):
"is_favorite",
"link_role",
"link_reach",
"nb_accesses",
"nb_accesses_ancestors",
"nb_accesses_direct",
"numchild",
"path",
"updated_at",
"user_roles",
"user_role",
]
def get_user_roles(self, document):
def to_representation(self, instance):
"""Precompute once per instance"""
paths_links_mapping = self.context.get("paths_links_mapping")
if paths_links_mapping is not None:
links = paths_links_mapping.get(instance.path[: -instance.steplen], [])
instance.ancestors_link_definition = choices.get_equivalent_link_definition(
links
)
return super().to_representation(instance)
def get_abilities(self, instance) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if not request:
return {}
return instance.get_abilities(request.user)
def get_user_role(self, instance):
"""
Return roles of the logged-in user for the current document,
taking into account ancestors.
"""
request = self.context.get("request")
if request:
return document.get_roles(request.user)
return []
return instance.get_role(request.user) if request else None
class DocumentLightSerializer(serializers.ModelSerializer):
"""Minial document serializer for nesting in document accesses."""
class Meta:
model = models.Document
fields = ["id", "path", "depth"]
read_only_fields = ["id", "path", "depth"]
class DocumentSerializer(ListDocumentSerializer):
@@ -206,6 +165,10 @@ class DocumentSerializer(ListDocumentSerializer):
fields = [
"id",
"abilities",
"ancestors_link_reach",
"ancestors_link_role",
"computed_link_reach",
"computed_link_role",
"content",
"created_at",
"creator",
@@ -214,27 +177,33 @@ class DocumentSerializer(ListDocumentSerializer):
"is_favorite",
"link_role",
"link_reach",
"nb_accesses",
"nb_accesses_ancestors",
"nb_accesses_direct",
"numchild",
"path",
"title",
"updated_at",
"user_roles",
"user_role",
]
read_only_fields = [
"id",
"abilities",
"ancestors_link_reach",
"ancestors_link_role",
"computed_link_reach",
"computed_link_role",
"created_at",
"creator",
"depth",
"is_favorite",
"link_role",
"link_reach",
"nb_accesses",
"nb_accesses_ancestors",
"nb_accesses_direct",
"numchild",
"path",
"updated_at",
"user_roles",
"user_role",
]
def get_fields(self):
@@ -260,6 +229,158 @@ class DocumentSerializer(ListDocumentSerializer):
return value
def validate_content(self, value):
"""Validate the content field."""
if not value:
return None
try:
b64decode(value, validate=True)
except binascii.Error as err:
raise serializers.ValidationError("Invalid base64 content.") from err
return value
def save(self, **kwargs):
"""
Process the content field to extract attachment keys and update the document's
"attachments" field for access control.
"""
content = self.validated_data.get("content", "")
extracted_attachments = set(utils.extract_attachments(content))
existing_attachments = (
set(self.instance.attachments or []) if self.instance else set()
)
new_attachments = extracted_attachments - existing_attachments
if new_attachments:
attachments_documents = (
models.Document.objects.filter(
attachments__overlap=list(new_attachments)
)
.only("path", "attachments")
.order_by("path")
)
user = self.context["request"].user
readable_per_se_paths = (
models.Document.objects.readable_per_se(user)
.order_by("path")
.values_list("path", flat=True)
)
readable_attachments_paths = utils.filter_descendants(
[doc.path for doc in attachments_documents],
readable_per_se_paths,
skip_sorting=True,
)
readable_attachments = set()
for document in attachments_documents:
if document.path not in readable_attachments_paths:
continue
readable_attachments.update(set(document.attachments) & new_attachments)
# Update attachments with readable keys
self.validated_data["attachments"] = list(
existing_attachments | readable_attachments
)
return super().save(**kwargs)
class DocumentAccessSerializer(serializers.ModelSerializer):
"""Serialize document accesses."""
document = DocumentLightSerializer(read_only=True)
user_id = serializers.PrimaryKeyRelatedField(
queryset=models.User.objects.all(),
write_only=True,
source="user",
required=False,
allow_null=True,
)
user = UserSerializer(read_only=True)
team = serializers.CharField(required=False, allow_blank=True)
abilities = serializers.SerializerMethodField(read_only=True)
max_ancestors_role = serializers.SerializerMethodField(read_only=True)
max_role = serializers.SerializerMethodField(read_only=True)
class Meta:
model = models.DocumentAccess
resource_field_name = "document"
fields = [
"id",
"document",
"user",
"user_id",
"team",
"role",
"abilities",
"max_ancestors_role",
"max_role",
]
read_only_fields = [
"id",
"document",
"abilities",
"max_ancestors_role",
"max_role",
]
def get_abilities(self, instance) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return instance.get_abilities(request.user)
return {}
def get_max_ancestors_role(self, instance):
"""Return max_ancestors_role if annotated; else None."""
return getattr(instance, "max_ancestors_role", None)
def get_max_role(self, instance):
"""Return max_ancestors_role if annotated; else None."""
return choices.RoleChoices.max(
getattr(instance, "max_ancestors_role", None),
instance.role,
)
def update(self, instance, validated_data):
"""Make "user" field readonly but only on update."""
validated_data.pop("team", None)
validated_data.pop("user", None)
return super().update(instance, validated_data)
class DocumentAccessLightSerializer(DocumentAccessSerializer):
"""Serialize document accesses with limited fields."""
user = UserLightSerializer(read_only=True)
class Meta:
model = models.DocumentAccess
resource_field_name = "document"
fields = [
"id",
"document",
"user",
"team",
"role",
"abilities",
"max_ancestors_role",
"max_role",
]
read_only_fields = [
"id",
"document",
"team",
"role",
"abilities",
"max_ancestors_role",
"max_role",
]
class ServerCreateDocumentSerializer(serializers.Serializer):
"""
@@ -359,7 +480,7 @@ class ServerCreateDocumentSerializer(serializers.Serializer):
raise NotImplementedError("Update is not supported for this serializer.")
class LinkDocumentSerializer(BaseResourceSerializer):
class LinkDocumentSerializer(serializers.ModelSerializer):
"""
Serialize link configuration for documents.
We expose it separately from document in order to simplify and secure access control.
@@ -373,6 +494,27 @@ class LinkDocumentSerializer(BaseResourceSerializer):
]
class DocumentDuplicationSerializer(serializers.Serializer):
"""
Serializer for duplicating a document.
Allows specifying whether to keep access permissions.
"""
with_accesses = serializers.BooleanField(default=False)
def create(self, validated_data):
"""
This serializer is not intended to create objects.
"""
raise NotImplementedError("This serializer does not support creation.")
def update(self, instance, validated_data):
"""
This serializer is not intended to update objects.
"""
raise NotImplementedError("This serializer does not support updating.")
# Suppress the warning about not implementing `create` and `update` methods
# since we don't use a model and only rely on the serializer for validation
# pylint: disable=abstract-method
@@ -396,16 +538,17 @@ class FileUploadSerializer(serializers.Serializer):
mime = magic.Magic(mime=True)
magic_mime_type = mime.from_buffer(file.read(1024))
file.seek(0) # Reset file pointer to the beginning after reading
self.context["is_unsafe"] = False
if settings.DOCUMENT_ATTACHMENT_CHECK_UNSAFE_MIME_TYPES_ENABLED:
self.context["is_unsafe"] = (
magic_mime_type in settings.DOCUMENT_UNSAFE_MIME_TYPES
)
self.context["is_unsafe"] = (
magic_mime_type in settings.DOCUMENT_UNSAFE_MIME_TYPES
)
extension_mime_type, _ = mimetypes.guess_type(file.name)
extension_mime_type, _ = mimetypes.guess_type(file.name)
# Try guessing a coherent extension from the mimetype
if extension_mime_type != magic_mime_type:
self.context["is_unsafe"] = True
# Try guessing a coherent extension from the mimetype
if extension_mime_type != magic_mime_type:
self.context["is_unsafe"] = True
guessed_ext = mimetypes.guess_extension(magic_mime_type)
# Missing extensions or extensions longer than 5 characters (it's as long as an extension
@@ -418,6 +561,7 @@ class FileUploadSerializer(serializers.Serializer):
self.context["expected_extension"] = extension
self.context["content_type"] = magic_mime_type
self.context["file_name"] = file.name
return file
@@ -426,12 +570,16 @@ class FileUploadSerializer(serializers.Serializer):
attrs["expected_extension"] = self.context["expected_extension"]
attrs["is_unsafe"] = self.context["is_unsafe"]
attrs["content_type"] = self.context["content_type"]
attrs["file_name"] = self.context["file_name"]
return attrs
class TemplateSerializer(BaseResourceSerializer):
class TemplateSerializer(serializers.ModelSerializer):
"""Serialize templates."""
abilities = serializers.SerializerMethodField(read_only=True)
accesses = TemplateAccessSerializer(many=True, read_only=True)
class Meta:
model = models.Template
fields = [
@@ -445,6 +593,13 @@ class TemplateSerializer(BaseResourceSerializer):
]
read_only_fields = ["id", "accesses", "abilities"]
def get_abilities(self, document) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return document.get_abilities(request.user)
return {}
# pylint: disable=abstract-method
class DocumentGenerationSerializer(serializers.Serializer):
@@ -531,6 +686,50 @@ class InvitationSerializer(serializers.ModelSerializer):
return role
class RoleSerializer(serializers.Serializer):
"""Serializer validating role choices."""
role = serializers.ChoiceField(
choices=models.RoleChoices.choices, required=False, allow_null=True
)
class DocumentAskForAccessCreateSerializer(serializers.Serializer):
"""Serializer for creating a document ask for access."""
role = serializers.ChoiceField(
choices=models.RoleChoices.choices,
required=False,
default=models.RoleChoices.READER,
)
class DocumentAskForAccessSerializer(serializers.ModelSerializer):
"""Serializer for document ask for access model"""
abilities = serializers.SerializerMethodField(read_only=True)
user = UserSerializer(read_only=True)
class Meta:
model = models.DocumentAskForAccess
fields = [
"id",
"document",
"user",
"role",
"created_at",
"abilities",
]
read_only_fields = ["id", "document", "user", "role", "created_at", "abilities"]
def get_abilities(self, invitation) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return invitation.get_abilities(request.user)
return {}
class VersionFilterSerializer(serializers.Serializer):
"""Validate version filters applied to the list endpoint."""

View File

@@ -11,6 +11,35 @@ import botocore
from rest_framework.throttling import BaseThrottle
def nest_tree(flat_list, steplen):
"""
Convert a flat list of serialized documents into a nested tree making advantage
of the`path` field and its step length.
"""
node_dict = {}
roots = []
# Sort the flat list by path to ensure parent nodes are processed first
flat_list.sort(key=lambda x: x["path"])
for node in flat_list:
node["children"] = [] # Initialize children list
node_dict[node["path"]] = node
# Determine parent path
parent_path = node["path"][:-steplen]
if parent_path in node_dict:
node_dict[parent_path]["children"].append(node)
else:
roots.append(node) # Collect root nodes
if len(roots) > 1:
raise ValueError("More than one root element detected.")
return roots[0] if roots else None
def filter_root_paths(paths, skip_sorting=False):
"""
Filters root paths from a list of paths representing a tree structure.

File diff suppressed because it is too large Load Diff

View File

@@ -1,130 +1,59 @@
"""Authentication Backends for the Impress core app."""
import logging
import os
from django.conf import settings
from django.core.exceptions import SuspiciousOperation
from django.utils.translation import gettext_lazy as _
import requests
from mozilla_django_oidc.auth import (
OIDCAuthenticationBackend as MozillaOIDCAuthenticationBackend,
from lasuite.oidc_login.backends import (
OIDCAuthenticationBackend as LaSuiteOIDCAuthenticationBackend,
)
from core.models import DuplicateEmailError, User
from core.models import DuplicateEmailError
logger = logging.getLogger(__name__)
# Settings renamed warnings
if os.environ.get("USER_OIDC_FIELDS_TO_FULLNAME"):
logger.warning(
"USER_OIDC_FIELDS_TO_FULLNAME has been renamed to "
"OIDC_USERINFO_FULLNAME_FIELDS please update your settings."
)
class OIDCAuthenticationBackend(MozillaOIDCAuthenticationBackend):
if os.environ.get("USER_OIDC_FIELD_TO_SHORTNAME"):
logger.warning(
"USER_OIDC_FIELD_TO_SHORTNAME has been renamed to "
"OIDC_USERINFO_SHORTNAME_FIELD please update your settings."
)
class OIDCAuthenticationBackend(LaSuiteOIDCAuthenticationBackend):
"""Custom OpenID Connect (OIDC) Authentication Backend.
This class overrides the default OIDC Authentication Backend to accommodate differences
in the User and Identity models, and handles signed and/or encrypted UserInfo response.
"""
def get_userinfo(self, access_token, id_token, payload):
"""Return user details dictionary.
def get_extra_claims(self, user_info):
"""
Return extra claims from user_info.
Parameters:
- access_token (str): The access token.
- id_token (str): The id token (unused).
- payload (dict): The token payload (unused).
Note: The id_token and payload parameters are unused in this implementation,
but were kept to preserve base method signature.
Note: It handles signed and/or encrypted UserInfo Response. It is required by
Agent Connect, which follows the OIDC standard. It forces us to override the
base method, which deal with 'application/json' response.
Args:
user_info (dict): The user information dictionary.
Returns:
- dict: User details dictionary obtained from the OpenID Connect user endpoint.
dict: A dictionary of extra claims.
"""
user_response = requests.get(
self.OIDC_OP_USER_ENDPOINT,
headers={"Authorization": f"Bearer {access_token}"},
verify=self.get_settings("OIDC_VERIFY_SSL", True),
timeout=self.get_settings("OIDC_TIMEOUT", None),
proxies=self.get_settings("OIDC_PROXY", None),
)
user_response.raise_for_status()
try:
userinfo = user_response.json()
except ValueError:
try:
userinfo = self.verify_token(user_response.text)
except Exception as e:
raise SuspiciousOperation(
_("Invalid response format or token verification failed")
) from e
return userinfo
def verify_claims(self, claims):
"""
Verify the presence of essential claims and the "sub" (which is mandatory as defined
by the OIDC specification) to decide if authentication should be allowed.
"""
essential_claims = settings.USER_OIDC_ESSENTIAL_CLAIMS
missing_claims = [claim for claim in essential_claims if claim not in claims]
if missing_claims:
logger.error("Missing essential claims: %s", missing_claims)
return False
return True
def get_or_create_user(self, access_token, id_token, payload):
"""Return a User based on userinfo. Create a new user if no match is found."""
user_info = self.get_userinfo(access_token, id_token, payload)
if not self.verify_claims(user_info):
raise SuspiciousOperation("Claims verification failed.")
sub = user_info["sub"]
email = user_info.get("email")
# Get user's full name from OIDC fields defined in settings
full_name = self.compute_full_name(user_info)
short_name = user_info.get(settings.USER_OIDC_FIELD_TO_SHORTNAME)
claims = {
"email": email,
"full_name": full_name,
"short_name": short_name,
return {
"full_name": self.compute_full_name(user_info),
"short_name": user_info.get(settings.OIDC_USERINFO_SHORTNAME_FIELD),
}
def get_existing_user(self, sub, email):
"""Fetch existing user by sub or email."""
try:
user = User.objects.get_user_by_sub_or_email(sub, email)
return self.UserModel.objects.get_user_by_sub_or_email(sub, email)
except DuplicateEmailError as err:
raise SuspiciousOperation(err.message) from err
if user:
if not user.is_active:
raise SuspiciousOperation(_("User account is disabled"))
self.update_user_if_needed(user, claims)
elif self.get_settings("OIDC_CREATE_USER", True):
user = User.objects.create(sub=sub, password="!", **claims) # noqa: S106
return user
def compute_full_name(self, user_info):
"""Compute user's full name based on OIDC fields in settings."""
name_fields = settings.USER_OIDC_FIELDS_TO_FULLNAME
full_name = " ".join(
user_info[field] for field in name_fields if user_info.get(field)
)
return full_name or None
def update_user_if_needed(self, user, claims):
"""Update user claims if they have changed."""
has_changed = any(
value and value != getattr(user, key) for key, value in claims.items()
)
if has_changed:
updated_claims = {key: value for key, value in claims.items() if value}
self.UserModel.objects.filter(id=user.id).update(**updated_claims)

View File

@@ -1,18 +0,0 @@
"""Authentication URLs for the People core app."""
from django.urls import path
from mozilla_django_oidc.urls import urlpatterns as mozzila_oidc_urls
from .views import OIDCLogoutCallbackView, OIDCLogoutView
urlpatterns = [
# Override the default 'logout/' path from Mozilla Django OIDC with our custom view.
path("logout/", OIDCLogoutView.as_view(), name="oidc_logout_custom"),
path(
"logout-callback/",
OIDCLogoutCallbackView.as_view(),
name="oidc_logout_callback",
),
*mozzila_oidc_urls,
]

View File

@@ -1,137 +0,0 @@
"""Authentication Views for the People core app."""
from urllib.parse import urlencode
from django.contrib import auth
from django.core.exceptions import SuspiciousOperation
from django.http import HttpResponseRedirect
from django.urls import reverse
from django.utils import crypto
from mozilla_django_oidc.utils import (
absolutify,
)
from mozilla_django_oidc.views import (
OIDCLogoutView as MozillaOIDCOIDCLogoutView,
)
class OIDCLogoutView(MozillaOIDCOIDCLogoutView):
"""Custom logout view for handling OpenID Connect (OIDC) logout flow.
Adds support for handling logout callbacks from the identity provider (OP)
by initiating the logout flow if the user has an active session.
The Django session is retained during the logout process to persist the 'state' OIDC parameter.
This parameter is crucial for maintaining the integrity of the logout flow between this call
and the subsequent callback.
"""
@staticmethod
def persist_state(request, state):
"""Persist the given 'state' parameter in the session's 'oidc_states' dictionary
This method is used to store the OIDC state parameter in the session, according to the
structure expected by Mozilla Django OIDC's 'add_state_and_verifier_and_nonce_to_session'
utility function.
"""
if "oidc_states" not in request.session or not isinstance(
request.session["oidc_states"], dict
):
request.session["oidc_states"] = {}
request.session["oidc_states"][state] = {}
request.session.save()
def construct_oidc_logout_url(self, request):
"""Create the redirect URL for interfacing with the OIDC provider.
Retrieves the necessary parameters from the session and constructs the URL
required to initiate logout with the OpenID Connect provider.
If no ID token is found in the session, the logout flow will not be initiated,
and the method will return the default redirect URL.
The 'state' parameter is generated randomly and persisted in the session to ensure
its integrity during the subsequent callback.
"""
oidc_logout_endpoint = self.get_settings("OIDC_OP_LOGOUT_ENDPOINT")
if not oidc_logout_endpoint:
return self.redirect_url
reverse_url = reverse("oidc_logout_callback")
id_token = request.session.get("oidc_id_token", None)
if not id_token:
return self.redirect_url
query = {
"id_token_hint": id_token,
"state": crypto.get_random_string(self.get_settings("OIDC_STATE_SIZE", 32)),
"post_logout_redirect_uri": absolutify(request, reverse_url),
}
self.persist_state(request, query["state"])
return f"{oidc_logout_endpoint}?{urlencode(query)}"
def post(self, request):
"""Handle user logout.
If the user is not authenticated, redirects to the default logout URL.
Otherwise, constructs the OIDC logout URL and redirects the user to start
the logout process.
If the user is redirected to the default logout URL, ensure her Django session
is terminated.
"""
logout_url = self.redirect_url
if request.user.is_authenticated:
logout_url = self.construct_oidc_logout_url(request)
# If the user is not redirected to the OIDC provider, ensure logout
if logout_url == self.redirect_url:
auth.logout(request)
return HttpResponseRedirect(logout_url)
class OIDCLogoutCallbackView(MozillaOIDCOIDCLogoutView):
"""Custom view for handling the logout callback from the OpenID Connect (OIDC) provider.
Handles the callback after logout from the identity provider (OP).
Verifies the state parameter and performs necessary logout actions.
The Django session is maintained during the logout process to ensure the integrity
of the logout flow initiated in the previous step.
"""
http_method_names = ["get"]
def get(self, request):
"""Handle the logout callback.
If the user is not authenticated, redirects to the default logout URL.
Otherwise, verifies the state parameter and performs necessary logout actions.
"""
if not request.user.is_authenticated:
return HttpResponseRedirect(self.redirect_url)
state = request.GET.get("state")
if state not in request.session.get("oidc_states", {}):
msg = "OIDC callback state not found in session `oidc_states`!"
raise SuspiciousOperation(msg)
del request.session["oidc_states"][state]
request.session.save()
auth.logout(request)
return HttpResponseRedirect(self.redirect_url)

115
src/backend/core/choices.py Normal file
View File

@@ -0,0 +1,115 @@
"""Declare and configure choices for Docs' core application."""
from django.db.models import TextChoices
from django.utils.translation import gettext_lazy as _
class PriorityTextChoices(TextChoices):
"""
This class inherits from Django's TextChoices and provides a method to get the priority
of a given value based on its position in the class.
"""
@classmethod
def get_priority(cls, role):
"""Returns the priority of the given role based on its order in the class."""
members = list(cls.__members__.values())
return members.index(role) + 1 if role in members else 0
@classmethod
def max(cls, *roles):
"""
Return the highest-priority role among the given roles, using get_priority().
If no valid roles are provided, returns None.
"""
valid_roles = [role for role in roles if cls.get_priority(role) is not None]
if not valid_roles:
return None
return max(valid_roles, key=cls.get_priority)
class LinkRoleChoices(PriorityTextChoices):
"""Defines the possible roles a link can offer on a document."""
READER = "reader", _("Reader") # Can read
EDITOR = "editor", _("Editor") # Can read and edit
class RoleChoices(PriorityTextChoices):
"""Defines the possible roles a user can have in a resource."""
READER = "reader", _("Reader") # Can read
EDITOR = "editor", _("Editor") # Can read and edit
ADMIN = "administrator", _("Administrator") # Can read, edit, delete and share
OWNER = "owner", _("Owner")
PRIVILEGED_ROLES = [RoleChoices.ADMIN, RoleChoices.OWNER]
class LinkReachChoices(PriorityTextChoices):
"""Defines types of access for links"""
RESTRICTED = (
"restricted",
_("Restricted"),
) # Only users with a specific access can read/edit the document
AUTHENTICATED = (
"authenticated",
_("Authenticated"),
) # Any authenticated user can access the document
PUBLIC = "public", _("Public") # Even anonymous users can access the document
@classmethod
def get_select_options(cls, link_reach, link_role):
"""
Determines the valid select options for link reach and link role depending on the
ancestors' link reach/role given as arguments.
Returns:
Dictionary mapping possible reach levels to their corresponding possible roles.
"""
return {
reach: [
role
for role in LinkRoleChoices.values
if LinkRoleChoices.get_priority(role)
>= LinkRoleChoices.get_priority(link_role)
]
if reach != cls.RESTRICTED
else None
for reach in cls.values
if LinkReachChoices.get_priority(reach)
>= LinkReachChoices.get_priority(link_reach)
}
def get_equivalent_link_definition(ancestors_links):
"""
Return the (reach, role) pair with:
1. Highest reach
2. Highest role among links having that reach
"""
if not ancestors_links:
return {"link_reach": None, "link_role": None}
# 1) Find the highest reach
max_reach = max(
ancestors_links,
key=lambda link: LinkReachChoices.get_priority(link["link_reach"]),
)["link_reach"]
# 2) Among those, find the highest role (ignore role if RESTRICTED)
if max_reach == LinkReachChoices.RESTRICTED:
max_role = None
else:
max_role = max(
(
link["link_role"]
for link in ancestors_links
if link["link_reach"] == max_reach
),
key=LinkRoleChoices.get_priority,
)
return {"link_reach": max_reach, "link_role": max_role}

View File

@@ -2,10 +2,27 @@
Core application enums declaration
"""
from django.conf import global_settings
import re
from enum import StrEnum
from django.conf import global_settings, settings
from django.db import models
from django.utils.translation import gettext_lazy as _
ATTACHMENTS_FOLDER = "attachments"
UUID_REGEX = (
r"[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}"
)
FILE_EXT_REGEX = r"\.[a-zA-Z0-9]{1,10}"
MEDIA_STORAGE_URL_PATTERN = re.compile(
f"{settings.MEDIA_URL:s}(?P<pk>{UUID_REGEX:s})/"
f"(?P<attachment>{ATTACHMENTS_FOLDER:s}/{UUID_REGEX:s}(?:-unsafe)?{FILE_EXT_REGEX:s})$"
)
MEDIA_STORAGE_URL_EXTRACT = re.compile(
f"{settings.MEDIA_URL:s}({UUID_REGEX}/{ATTACHMENTS_FOLDER}/{UUID_REGEX}{FILE_EXT_REGEX})"
)
# In Django's code base, `LANGUAGES` is set by default with all supported languages.
# We can use it for the choice of languages which should not be limited to the few languages
# active in the app.
@@ -22,3 +39,10 @@ class MoveNodePositionChoices(models.TextChoices):
LAST_SIBLING = "last-sibling", _("Last sibling")
LEFT = "left", _("Left")
RIGHT = "right", _("Right")
class DocumentAttachmentStatus(StrEnum):
"""Defines the possible statuses for an attachment."""
PROCESSING = "processing"
READY = "ready"

View File

@@ -1,4 +1,3 @@
# ruff: noqa: S311
"""
Core application factories
"""
@@ -13,12 +12,30 @@ from core import models
fake = Faker()
YDOC_HELLO_WORLD_BASE64 = (
"AR717vLVDgAHAQ5kb2N1bWVudC1zdG9yZQMKYmxvY2tHcm91cAcA9e7y1Q4AAw5ibG9ja0NvbnRh"
"aW5lcgcA9e7y1Q4BAwdoZWFkaW5nBwD17vLVDgIGBgD17vLVDgMGaXRhbGljAnt9hPXu8tUOBAVI"
"ZWxsb4b17vLVDgkGaXRhbGljBG51bGwoAPXu8tUOAg10ZXh0QWxpZ25tZW50AXcEbGVmdCgA9e7y"
"1Q4CBWxldmVsAX0BKAD17vLVDgECaWQBdyQwNGQ2MjM0MS04MzI2LTQyMzYtYTA4My00ODdlMjZm"
"YWQyMzAoAPXu8tUOAQl0ZXh0Q29sb3IBdwdkZWZhdWx0KAD17vLVDgEPYmFja2dyb3VuZENvbG9y"
"AXcHZGVmYXVsdIf17vLVDgEDDmJsb2NrQ29udGFpbmVyBwD17vLVDhADDmJ1bGxldExpc3RJdGVt"
"BwD17vLVDhEGBAD17vLVDhIBd4b17vLVDhMEYm9sZAJ7fYT17vLVDhQCb3KG9e7y1Q4WBGJvbGQE"
"bnVsbIT17vLVDhcCbGQoAPXu8tUOEQ10ZXh0QWxpZ25tZW50AXcEbGVmdCgA9e7y1Q4QAmlkAXck"
"ZDM1MWUwNjgtM2U1NS00MjI2LThlYTUtYWJiMjYzMTk4ZTJhKAD17vLVDhAJdGV4dENvbG9yAXcH"
"ZGVmYXVsdCgA9e7y1Q4QD2JhY2tncm91bmRDb2xvcgF3B2RlZmF1bHSH9e7y1Q4QAw5ibG9ja0Nv"
"bnRhaW5lcgcA9e7y1Q4eAwlwYXJhZ3JhcGgoAPXu8tUOHw10ZXh0QWxpZ25tZW50AXcEbGVmdCgA"
"9e7y1Q4eAmlkAXckODk3MDBjMDctZTBlMS00ZmUwLWFjYTItODQ5MzIwOWE3ZTQyKAD17vLVDh4J"
"dGV4dENvbG9yAXcHZGVmYXVsdCgA9e7y1Q4eD2JhY2tncm91bmRDb2xvcgF3B2RlZmF1bHQA"
)
class UserFactory(factory.django.DjangoModelFactory):
"""A factory to random users for testing purposes."""
class Meta:
model = models.User
# Skip postgeneration save, no save is made in the postgeneration methods.
skip_postgeneration_save = True
sub = factory.Sequence(lambda n: f"user{n!s}")
email = factory.Faker("email")
@@ -75,7 +92,7 @@ class DocumentFactory(factory.django.DjangoModelFactory):
title = factory.Sequence(lambda n: f"document{n}")
excerpt = factory.Sequence(lambda n: f"excerpt{n}")
content = factory.Sequence(lambda n: f"content{n}")
content = YDOC_HELLO_WORLD_BASE64
creator = factory.SubFactory(UserFactory)
deleted_at = None
link_reach = factory.fuzzy.FuzzyChoice(
@@ -165,6 +182,17 @@ class TeamDocumentAccessFactory(factory.django.DjangoModelFactory):
role = factory.fuzzy.FuzzyChoice([r[0] for r in models.RoleChoices.choices])
class DocumentAskForAccessFactory(factory.django.DjangoModelFactory):
"""Create fake document ask for access for testing."""
class Meta:
model = models.DocumentAskForAccess
document = factory.SubFactory(DocumentFactory)
user = factory.SubFactory(UserFactory)
role = factory.fuzzy.FuzzyChoice([r[0] for r in models.RoleChoices.choices])
class TemplateFactory(factory.django.DjangoModelFactory):
"""A factory to create templates"""

View File

@@ -0,0 +1,52 @@
"""Malware detection callbacks"""
import logging
from django.core.files.storage import default_storage
from lasuite.malware_detection.enums import ReportStatus
from core.enums import DocumentAttachmentStatus
from core.models import Document
logger = logging.getLogger(__name__)
security_logger = logging.getLogger("docs.security")
def malware_detection_callback(file_path, status, error_info, **kwargs):
"""Malware detection callback"""
if status == ReportStatus.SAFE:
logger.info("File %s is safe", file_path)
# Get existing metadata
s3_client = default_storage.connection.meta.client
bucket_name = default_storage.bucket_name
head_resp = s3_client.head_object(Bucket=bucket_name, Key=file_path)
metadata = head_resp.get("Metadata", {})
metadata.update({"status": DocumentAttachmentStatus.READY})
# Update status in metadata
s3_client.copy_object(
Bucket=bucket_name,
CopySource={"Bucket": bucket_name, "Key": file_path},
Key=file_path,
ContentType=head_resp.get("ContentType"),
Metadata=metadata,
MetadataDirective="REPLACE",
)
return
document_id = kwargs.get("document_id")
security_logger.warning(
"File %s for document %s is infected with malware. Error info: %s",
file_path,
document_id,
error_info,
)
# Remove the file from the document and change the status to unsafe
document = Document.objects.get(pk=document_id)
document.attachments.remove(file_path)
document.save(update_fields=["attachments"])
# Delete the file from the storage
default_storage.delete(file_path)

View File

@@ -1,166 +1,552 @@
# Generated by Django 5.0.3 on 2024-05-28 20:29
import uuid
import django.contrib.auth.models
import django.core.validators
import django.db.models.deletion
import timezone_field.fields
import uuid
from django.conf import settings
from django.db import migrations, models
import timezone_field.fields
class Migration(migrations.Migration):
initial = True
dependencies = [
('auth', '0012_alter_user_first_name_max_length'),
("auth", "0012_alter_user_first_name_max_length"),
]
operations = [
migrations.CreateModel(
name='Document',
name="Document",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('title', models.CharField(max_length=255, verbose_name='title')),
('is_public', models.BooleanField(default=False, help_text='Whether this document is public for anyone to use.', verbose_name='public')),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
("title", models.CharField(max_length=255, verbose_name="title")),
(
"is_public",
models.BooleanField(
default=False,
help_text="Whether this document is public for anyone to use.",
verbose_name="public",
),
),
],
options={
'verbose_name': 'Document',
'verbose_name_plural': 'Documents',
'db_table': 'impress_document',
'ordering': ('title',),
"verbose_name": "Document",
"verbose_name_plural": "Documents",
"db_table": "impress_document",
"ordering": ("title",),
},
),
migrations.CreateModel(
name='Template',
name="Template",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('title', models.CharField(max_length=255, verbose_name='title')),
('description', models.TextField(blank=True, verbose_name='description')),
('code', models.TextField(blank=True, verbose_name='code')),
('css', models.TextField(blank=True, verbose_name='css')),
('is_public', models.BooleanField(default=False, help_text='Whether this template is public for anyone to use.', verbose_name='public')),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
("title", models.CharField(max_length=255, verbose_name="title")),
(
"description",
models.TextField(blank=True, verbose_name="description"),
),
("code", models.TextField(blank=True, verbose_name="code")),
("css", models.TextField(blank=True, verbose_name="css")),
(
"is_public",
models.BooleanField(
default=False,
help_text="Whether this template is public for anyone to use.",
verbose_name="public",
),
),
],
options={
'verbose_name': 'Template',
'verbose_name_plural': 'Templates',
'db_table': 'impress_template',
'ordering': ('title',),
"verbose_name": "Template",
"verbose_name_plural": "Templates",
"db_table": "impress_template",
"ordering": ("title",),
},
),
migrations.CreateModel(
name='User',
name="User",
fields=[
('password', models.CharField(max_length=128, verbose_name='password')),
('last_login', models.DateTimeField(blank=True, null=True, verbose_name='last login')),
('is_superuser', models.BooleanField(default=False, help_text='Designates that this user has all permissions without explicitly assigning them.', verbose_name='superuser status')),
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('sub', models.CharField(blank=True, help_text='Required. 255 characters or fewer. Letters, numbers, and @/./+/-/_ characters only.', max_length=255, null=True, unique=True, validators=[django.core.validators.RegexValidator(message='Enter a valid sub. This value may contain only letters, numbers, and @/./+/-/_ characters.', regex='^[\\w.@+-]+\\Z')], verbose_name='sub')),
('email', models.EmailField(blank=True, max_length=254, null=True, verbose_name='identity email address')),
('admin_email', models.EmailField(blank=True, max_length=254, null=True, unique=True, verbose_name='admin email address')),
('language', models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language')),
('timezone', timezone_field.fields.TimeZoneField(choices_display='WITH_GMT_OFFSET', default='UTC', help_text='The timezone in which the user wants to see times.', use_pytz=False)),
('is_device', models.BooleanField(default=False, help_text='Whether the user is a device or a real user.', verbose_name='device')),
('is_staff', models.BooleanField(default=False, help_text='Whether the user can log into this admin site.', verbose_name='staff status')),
('is_active', models.BooleanField(default=True, help_text='Whether this user should be treated as active. Unselect this instead of deleting accounts.', verbose_name='active')),
('groups', models.ManyToManyField(blank=True, help_text='The groups this user belongs to. A user will get all permissions granted to each of their groups.', related_name='user_set', related_query_name='user', to='auth.group', verbose_name='groups')),
('user_permissions', models.ManyToManyField(blank=True, help_text='Specific permissions for this user.', related_name='user_set', related_query_name='user', to='auth.permission', verbose_name='user permissions')),
("password", models.CharField(max_length=128, verbose_name="password")),
(
"last_login",
models.DateTimeField(
blank=True, null=True, verbose_name="last login"
),
),
(
"is_superuser",
models.BooleanField(
default=False,
help_text="Designates that this user has all permissions without explicitly assigning them.",
verbose_name="superuser status",
),
),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
(
"sub",
models.CharField(
blank=True,
help_text="Required. 255 characters or fewer. Letters, numbers, and @/./+/-/_ characters only.",
max_length=255,
null=True,
unique=True,
validators=[
django.core.validators.RegexValidator(
message="Enter a valid sub. This value may contain only letters, numbers, and @/./+/-/_ characters.",
regex="^[\\w.@+-]+\\Z",
)
],
verbose_name="sub",
),
),
(
"email",
models.EmailField(
blank=True,
max_length=254,
null=True,
verbose_name="identity email address",
),
),
(
"admin_email",
models.EmailField(
blank=True,
max_length=254,
null=True,
unique=True,
verbose_name="admin email address",
),
),
(
"language",
models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
(
"timezone",
timezone_field.fields.TimeZoneField(
choices_display="WITH_GMT_OFFSET",
default="UTC",
help_text="The timezone in which the user wants to see times.",
use_pytz=False,
),
),
(
"is_device",
models.BooleanField(
default=False,
help_text="Whether the user is a device or a real user.",
verbose_name="device",
),
),
(
"is_staff",
models.BooleanField(
default=False,
help_text="Whether the user can log into this admin site.",
verbose_name="staff status",
),
),
(
"is_active",
models.BooleanField(
default=True,
help_text="Whether this user should be treated as active. Unselect this instead of deleting accounts.",
verbose_name="active",
),
),
(
"groups",
models.ManyToManyField(
blank=True,
help_text="The groups this user belongs to. A user will get all permissions granted to each of their groups.",
related_name="user_set",
related_query_name="user",
to="auth.group",
verbose_name="groups",
),
),
(
"user_permissions",
models.ManyToManyField(
blank=True,
help_text="Specific permissions for this user.",
related_name="user_set",
related_query_name="user",
to="auth.permission",
verbose_name="user permissions",
),
),
],
options={
'verbose_name': 'user',
'verbose_name_plural': 'users',
'db_table': 'impress_user',
"verbose_name": "user",
"verbose_name_plural": "users",
"db_table": "impress_user",
},
managers=[
('objects', django.contrib.auth.models.UserManager()),
("objects", django.contrib.auth.models.UserManager()),
],
),
migrations.CreateModel(
name='DocumentAccess',
name="DocumentAccess",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('team', models.CharField(blank=True, max_length=100)),
('role', models.CharField(choices=[('reader', 'Reader'), ('editor', 'Editor'), ('administrator', 'Administrator'), ('owner', 'Owner')], default='reader', max_length=20)),
('document', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='accesses', to='core.document')),
('user', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
("team", models.CharField(blank=True, max_length=100)),
(
"role",
models.CharField(
choices=[
("reader", "Reader"),
("editor", "Editor"),
("administrator", "Administrator"),
("owner", "Owner"),
],
default="reader",
max_length=20,
),
),
(
"document",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="accesses",
to="core.document",
),
),
(
"user",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.CASCADE,
to=settings.AUTH_USER_MODEL,
),
),
],
options={
'verbose_name': 'Document/user relation',
'verbose_name_plural': 'Document/user relations',
'db_table': 'impress_document_access',
'ordering': ('-created_at',),
"verbose_name": "Document/user relation",
"verbose_name_plural": "Document/user relations",
"db_table": "impress_document_access",
"ordering": ("-created_at",),
},
),
migrations.CreateModel(
name='Invitation',
name="Invitation",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('email', models.EmailField(max_length=254, verbose_name='email address')),
('role', models.CharField(choices=[('reader', 'Reader'), ('editor', 'Editor'), ('administrator', 'Administrator'), ('owner', 'Owner')], default='reader', max_length=20)),
('document', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='invitations', to='core.document')),
('issuer', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='invitations', to=settings.AUTH_USER_MODEL)),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
(
"email",
models.EmailField(max_length=254, verbose_name="email address"),
),
(
"role",
models.CharField(
choices=[
("reader", "Reader"),
("editor", "Editor"),
("administrator", "Administrator"),
("owner", "Owner"),
],
default="reader",
max_length=20,
),
),
(
"document",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="invitations",
to="core.document",
),
),
(
"issuer",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="invitations",
to=settings.AUTH_USER_MODEL,
),
),
],
options={
'verbose_name': 'Document invitation',
'verbose_name_plural': 'Document invitations',
'db_table': 'impress_invitation',
"verbose_name": "Document invitation",
"verbose_name_plural": "Document invitations",
"db_table": "impress_invitation",
},
),
migrations.CreateModel(
name='TemplateAccess',
name="TemplateAccess",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('team', models.CharField(blank=True, max_length=100)),
('role', models.CharField(choices=[('reader', 'Reader'), ('editor', 'Editor'), ('administrator', 'Administrator'), ('owner', 'Owner')], default='reader', max_length=20)),
('template', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='accesses', to='core.template')),
('user', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
("team", models.CharField(blank=True, max_length=100)),
(
"role",
models.CharField(
choices=[
("reader", "Reader"),
("editor", "Editor"),
("administrator", "Administrator"),
("owner", "Owner"),
],
default="reader",
max_length=20,
),
),
(
"template",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="accesses",
to="core.template",
),
),
(
"user",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.CASCADE,
to=settings.AUTH_USER_MODEL,
),
),
],
options={
'verbose_name': 'Template/user relation',
'verbose_name_plural': 'Template/user relations',
'db_table': 'impress_template_access',
'ordering': ('-created_at',),
"verbose_name": "Template/user relation",
"verbose_name_plural": "Template/user relations",
"db_table": "impress_template_access",
"ordering": ("-created_at",),
},
),
migrations.AddConstraint(
model_name='documentaccess',
constraint=models.UniqueConstraint(condition=models.Q(('user__isnull', False)), fields=('user', 'document'), name='unique_document_user', violation_error_message='This user is already in this document.'),
model_name="documentaccess",
constraint=models.UniqueConstraint(
condition=models.Q(("user__isnull", False)),
fields=("user", "document"),
name="unique_document_user",
violation_error_message="This user is already in this document.",
),
),
migrations.AddConstraint(
model_name='documentaccess',
constraint=models.UniqueConstraint(condition=models.Q(('team__gt', '')), fields=('team', 'document'), name='unique_document_team', violation_error_message='This team is already in this document.'),
model_name="documentaccess",
constraint=models.UniqueConstraint(
condition=models.Q(("team__gt", "")),
fields=("team", "document"),
name="unique_document_team",
violation_error_message="This team is already in this document.",
),
),
migrations.AddConstraint(
model_name='documentaccess',
constraint=models.CheckConstraint(check=models.Q(models.Q(('team', ''), ('user__isnull', False)), models.Q(('team__gt', ''), ('user__isnull', True)), _connector='OR'), name='check_document_access_either_user_or_team', violation_error_message='Either user or team must be set, not both.'),
model_name="documentaccess",
constraint=models.CheckConstraint(
condition=models.Q(
models.Q(("team", ""), ("user__isnull", False)),
models.Q(("team__gt", ""), ("user__isnull", True)),
_connector="OR",
),
name="check_document_access_either_user_or_team",
violation_error_message="Either user or team must be set, not both.",
),
),
migrations.AddConstraint(
model_name='invitation',
constraint=models.UniqueConstraint(fields=('email', 'document'), name='email_and_document_unique_together'),
model_name="invitation",
constraint=models.UniqueConstraint(
fields=("email", "document"), name="email_and_document_unique_together"
),
),
migrations.AddConstraint(
model_name='templateaccess',
constraint=models.UniqueConstraint(condition=models.Q(('user__isnull', False)), fields=('user', 'template'), name='unique_template_user', violation_error_message='This user is already in this template.'),
model_name="templateaccess",
constraint=models.UniqueConstraint(
condition=models.Q(("user__isnull", False)),
fields=("user", "template"),
name="unique_template_user",
violation_error_message="This user is already in this template.",
),
),
migrations.AddConstraint(
model_name='templateaccess',
constraint=models.UniqueConstraint(condition=models.Q(('team__gt', '')), fields=('team', 'template'), name='unique_template_team', violation_error_message='This team is already in this template.'),
model_name="templateaccess",
constraint=models.UniqueConstraint(
condition=models.Q(("team__gt", "")),
fields=("team", "template"),
name="unique_template_team",
violation_error_message="This team is already in this template.",
),
),
migrations.AddConstraint(
model_name='templateaccess',
constraint=models.CheckConstraint(check=models.Q(models.Q(('team', ''), ('user__isnull', False)), models.Q(('team__gt', ''), ('user__isnull', True)), _connector='OR'), name='check_template_access_either_user_or_team', violation_error_message='Either user or team must be set, not both.'),
model_name="templateaccess",
constraint=models.CheckConstraint(
condition=models.Q(
models.Q(("team", ""), ("user__isnull", False)),
models.Q(("team__gt", ""), ("user__isnull", True)),
_connector="OR",
),
name="check_template_access_either_user_or_team",
violation_error_message="Either user or team must be set, not both.",
),
),
]

View File

@@ -1,9 +1,9 @@
from django.db import migrations
class Migration(migrations.Migration):
class Migration(migrations.Migration):
dependencies = [
('core', '0001_initial'),
("core", "0001_initial"),
]
operations = [

View File

@@ -1,52 +1,114 @@
# Generated by Django 5.1 on 2024-09-08 16:55
import django.db.models.deletion
import uuid
import django.db.models.deletion
from django.conf import settings
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0002_create_pg_trgm_extension'),
("core", "0002_create_pg_trgm_extension"),
]
operations = [
migrations.AddField(
model_name='document',
name='link_reach',
field=models.CharField(choices=[('restricted', 'Restricted'), ('authenticated', 'Authenticated'), ('public', 'Public')], default='authenticated', max_length=20),
model_name="document",
name="link_reach",
field=models.CharField(
choices=[
("restricted", "Restricted"),
("authenticated", "Authenticated"),
("public", "Public"),
],
default="authenticated",
max_length=20,
),
),
migrations.AddField(
model_name='document',
name='link_role',
field=models.CharField(choices=[('reader', 'Reader'), ('editor', 'Editor')], default='reader', max_length=20),
model_name="document",
name="link_role",
field=models.CharField(
choices=[("reader", "Reader"), ("editor", "Editor")],
default="reader",
max_length=20,
),
),
migrations.AlterField(
model_name='document',
name='is_public',
model_name="document",
name="is_public",
field=models.BooleanField(null=True),
),
migrations.AlterField(
model_name='user',
name='language',
field=models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language'),
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
migrations.CreateModel(
name='LinkTrace',
name="LinkTrace",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('document', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='link_traces', to='core.document')),
('user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='link_traces', to=settings.AUTH_USER_MODEL)),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
(
"document",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="link_traces",
to="core.document",
),
),
(
"user",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="link_traces",
to=settings.AUTH_USER_MODEL,
),
),
],
options={
'verbose_name': 'Document/user link trace',
'verbose_name_plural': 'Document/user link traces',
'db_table': 'impress_link_trace',
'constraints': [models.UniqueConstraint(fields=('user', 'document'), name='unique_link_trace_document_user', violation_error_message='A link trace already exists for this document/user.')],
"verbose_name": "Document/user link trace",
"verbose_name_plural": "Document/user link traces",
"db_table": "impress_link_trace",
"constraints": [
models.UniqueConstraint(
fields=("user", "document"),
name="unique_link_trace_document_user",
violation_error_message="A link trace already exists for this document/user.",
)
],
},
),
]

View File

@@ -1,13 +1,14 @@
# Generated by Django 5.1 on 2024-09-08 17:04
from django.db import migrations
def migrate_is_public_to_link_reach(apps, schema_editor):
"""
Forward migration: Migrate 'is_public' to 'link_reach'.
If is_public == True, set link_reach to 'public'
"""
Document = apps.get_model('core', 'Document')
Document.objects.filter(is_public=True).update(link_reach='public')
Document = apps.get_model("core", "Document")
Document.objects.filter(is_public=True).update(link_reach="public")
def reverse_migrate_link_reach_to_is_public(apps, schema_editor):
@@ -16,20 +17,20 @@ def reverse_migrate_link_reach_to_is_public(apps, schema_editor):
- If link_reach == 'public', set is_public to True
- Else set is_public to False
"""
Document = apps.get_model('core', 'Document')
Document.objects.filter(link_reach='public').update(is_public=True)
Document.objects.filter(link_reach__in=['restricted', "authenticated"]).update(is_public=False)
Document = apps.get_model("core", "Document")
Document.objects.filter(link_reach="public").update(is_public=True)
Document.objects.filter(link_reach__in=["restricted", "authenticated"]).update(
is_public=False
)
class Migration(migrations.Migration):
dependencies = [
('core', '0003_document_link_reach_document_link_role_and_more'),
("core", "0003_document_link_reach_document_link_role_and_more"),
]
operations = [
migrations.RunPython(
migrate_is_public_to_link_reach,
reverse_migrate_link_reach_to_is_public
migrate_is_public_to_link_reach, reverse_migrate_link_reach_to_is_public
),
]

View File

@@ -4,15 +4,16 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0004_migrate_is_public_to_link_reach'),
("core", "0004_migrate_is_public_to_link_reach"),
]
operations = [
migrations.AlterField(
model_name='document',
name='title',
field=models.CharField(blank=True, max_length=255, null=True, verbose_name='title'),
model_name="document",
name="title",
field=models.CharField(
blank=True, max_length=255, null=True, verbose_name="title"
),
),
]

View File

@@ -4,25 +4,34 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0005_remove_document_is_public_alter_document_link_reach_and_more'),
("core", "0005_remove_document_is_public_alter_document_link_reach_and_more"),
]
operations = [
migrations.AddField(
model_name='user',
name='full_name',
field=models.CharField(blank=True, max_length=100, null=True, verbose_name='full name'),
model_name="user",
name="full_name",
field=models.CharField(
blank=True, max_length=100, null=True, verbose_name="full name"
),
),
migrations.AddField(
model_name='user',
name='short_name',
field=models.CharField(blank=True, max_length=20, null=True, verbose_name='short name'),
model_name="user",
name="short_name",
field=models.CharField(
blank=True, max_length=20, null=True, verbose_name="short name"
),
),
migrations.AlterField(
model_name='user',
name='language',
field=models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language'),
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
]

View File

@@ -117,10 +117,10 @@ BEGIN
END $$;
"""
class Migration(migrations.Migration):
class Migration(migrations.Migration):
dependencies = [
('core', '0006_add_user_full_name_and_short_name'),
("core", "0006_add_user_full_name_and_short_name"),
]
operations = [

View File

@@ -4,15 +4,22 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0007_fix_users_duplicate'),
("core", "0007_fix_users_duplicate"),
]
operations = [
migrations.AlterField(
model_name='document',
name='link_reach',
field=models.CharField(choices=[('restricted', 'Restricted'), ('authenticated', 'Authenticated'), ('public', 'Public')], default='restricted', max_length=20),
model_name="document",
name="link_reach",
field=models.CharField(
choices=[
("restricted", "Restricted"),
("authenticated", "Authenticated"),
("public", "Public"),
],
default="restricted",
max_length=20,
),
),
]

View File

@@ -1,37 +1,87 @@
# Generated by Django 5.1.2 on 2024-11-08 07:59
import django.db.models.deletion
import uuid
import django.db.models.deletion
from django.conf import settings
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0008_alter_document_link_reach'),
("core", "0008_alter_document_link_reach"),
]
operations = [
migrations.AlterField(
model_name='user',
name='language',
field=models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language'),
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
migrations.CreateModel(
name='DocumentFavorite',
name="DocumentFavorite",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('document', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='favorited_by_users', to='core.document')),
('user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='favorite_documents', to=settings.AUTH_USER_MODEL)),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
(
"document",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="favorited_by_users",
to="core.document",
),
),
(
"user",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="favorite_documents",
to=settings.AUTH_USER_MODEL,
),
),
],
options={
'verbose_name': 'Document favorite',
'verbose_name_plural': 'Document favorites',
'db_table': 'impress_document_favorite',
'constraints': [models.UniqueConstraint(fields=('user', 'document'), name='unique_document_favorite_user', violation_error_message='This document is already targeted by a favorite relation instance for the same user.')],
"verbose_name": "Document favorite",
"verbose_name_plural": "Document favorites",
"db_table": "impress_document_favorite",
"constraints": [
models.UniqueConstraint(
fields=("user", "document"),
name="unique_document_favorite_user",
violation_error_message="This document is already targeted by a favorite relation instance for the same user.",
)
],
},
),
]

View File

@@ -7,25 +7,48 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0009_add_document_favorite'),
("core", "0009_add_document_favorite"),
]
operations = [
migrations.AddField(
model_name='document',
name='creator',
field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.RESTRICT, related_name='documents_created', to=settings.AUTH_USER_MODEL),
model_name="document",
name="creator",
field=models.ForeignKey(
null=True,
on_delete=django.db.models.deletion.RESTRICT,
related_name="documents_created",
to=settings.AUTH_USER_MODEL,
),
),
migrations.AlterField(
model_name='user',
name='language',
field=models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language'),
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
migrations.AlterField(
model_name='user',
name='sub',
field=models.CharField(blank=True, help_text='Required. 255 characters or fewer. Letters, numbers, and @/./+/-/_/: characters only.', max_length=255, null=True, unique=True, validators=[django.core.validators.RegexValidator(message='Enter a valid sub. This value may contain only letters, numbers, and @/./+/-/_/: characters.', regex='^[\\w.@+-:]+\\Z')], verbose_name='sub'),
model_name="user",
name="sub",
field=models.CharField(
blank=True,
help_text="Required. 255 characters or fewer. Letters, numbers, and @/./+/-/_/: characters only.",
max_length=255,
null=True,
unique=True,
validators=[
django.core.validators.RegexValidator(
message="Enter a valid sub. This value may contain only letters, numbers, and @/./+/-/_/: characters.",
regex="^[\\w.@+-:]+\\Z",
)
],
verbose_name="sub",
),
),
]

View File

@@ -3,7 +3,7 @@
import django.db.models.deletion
from django.conf import settings
from django.db import migrations
from django.db.models import F, ForeignKey, Subquery, OuterRef, Q
from django.db.models import F, ForeignKey, OuterRef, Q, Subquery
def set_creator_from_document_access(apps, schema_editor):
@@ -25,28 +25,37 @@ def set_creator_from_document_access(apps, schema_editor):
DocumentAccess = apps.get_model("core", "DocumentAccess")
# Update `creator` using the "owner" role
owner_subquery = DocumentAccess.objects.filter(
document=OuterRef('pk'),
user__isnull=False,
role='owner',
).order_by('created_at').values('user_id')[:1]
owner_subquery = (
DocumentAccess.objects.filter(
document=OuterRef("pk"),
user__isnull=False,
role="owner",
)
.order_by("created_at")
.values("user_id")[:1]
)
Document.objects.filter(
creator__isnull=True
).update(creator=Subquery(owner_subquery))
Document.objects.filter(creator__isnull=True).update(
creator=Subquery(owner_subquery)
)
class Migration(migrations.Migration):
dependencies = [
('core', '0010_add_field_creator_to_document'),
("core", "0010_add_field_creator_to_document"),
]
operations = [
migrations.RunPython(set_creator_from_document_access, reverse_code=migrations.RunPython.noop),
migrations.RunPython(
set_creator_from_document_access, reverse_code=migrations.RunPython.noop
),
migrations.AlterField(
model_name='document',
name='creator',
field=ForeignKey(on_delete=django.db.models.deletion.RESTRICT, related_name='documents_created', to=settings.AUTH_USER_MODEL),
model_name="document",
name="creator",
field=ForeignKey(
on_delete=django.db.models.deletion.RESTRICT,
related_name="documents_created",
to=settings.AUTH_USER_MODEL,
),
),
]

View File

@@ -6,25 +6,42 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0011_populate_creator_field_and_make_it_required'),
("core", "0011_populate_creator_field_and_make_it_required"),
]
operations = [
migrations.AlterField(
model_name='document',
name='creator',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.RESTRICT, related_name='documents_created', to=settings.AUTH_USER_MODEL),
model_name="document",
name="creator",
field=models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.RESTRICT,
related_name="documents_created",
to=settings.AUTH_USER_MODEL,
),
),
migrations.AlterField(
model_name='invitation',
name='issuer',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='invitations', to=settings.AUTH_USER_MODEL),
model_name="invitation",
name="issuer",
field=models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.CASCADE,
related_name="invitations",
to=settings.AUTH_USER_MODEL,
),
),
migrations.AlterField(
model_name='user',
name='language',
field=models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language'),
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
]

View File

@@ -2,10 +2,10 @@
from django.db import migrations
class Migration(migrations.Migration):
class Migration(migrations.Migration):
dependencies = [
('core', '0012_make_document_creator_and_invitation_issuer_optional'),
("core", "0012_make_document_creator_and_invitation_issuer_optional"),
]
operations = [

View File

@@ -4,28 +4,29 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0013_activate_fuzzystrmatch_extension'),
("core", "0013_activate_fuzzystrmatch_extension"),
]
operations = [
migrations.AddField(
model_name='document',
name='depth',
model_name="document",
name="depth",
field=models.PositiveIntegerField(default=0),
preserve_default=False,
),
migrations.AddField(
model_name='document',
name='numchild',
model_name="document",
name="numchild",
field=models.PositiveIntegerField(default=0),
),
migrations.AddField(
model_name='document',
name='path',
model_name="document",
name="path",
# Allow null values pending the next datamigration to populate the field
field=models.CharField(db_collation='C', max_length=252, null=True, unique=True),
field=models.CharField(
db_collation="C", max_length=252, null=True, unique=True
),
preserve_default=False,
),
]

View File

@@ -7,9 +7,10 @@ from treebeard.numconv import NumConv
ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
STEPLEN = 7
def set_path_on_existing_documents(apps, schema_editor):
"""
Updates the `path` and `depth` fields for all existing Document records
Updates the `path` and `depth` fields for all existing Document records
to ensure valid materialized paths.
This function assigns a unique `path` to each Document as a root node
@@ -26,27 +27,25 @@ def set_path_on_existing_documents(apps, schema_editor):
updates = []
for i, pk in enumerate(documents):
key = numconv.int2str(i)
path = "{0}{1}".format(
ALPHABET[0] * (STEPLEN - len(key)),
key
)
path = "{0}{1}".format(ALPHABET[0] * (STEPLEN - len(key)), key)
updates.append(Document(pk=pk, path=path, depth=1))
# Bulk update using the prepared updates list
Document.objects.bulk_update(updates, ['depth', 'path'])
Document.objects.bulk_update(updates, ["depth", "path"])
class Migration(migrations.Migration):
dependencies = [
('core', '0014_add_tree_structure_to_documents'),
("core", "0014_add_tree_structure_to_documents"),
]
operations = [
migrations.RunPython(set_path_on_existing_documents, reverse_code=migrations.RunPython.noop),
migrations.RunPython(
set_path_on_existing_documents, reverse_code=migrations.RunPython.noop
),
migrations.AlterField(
model_name='document',
name='path',
field=models.CharField(db_collation='C', max_length=252, unique=True),
model_name="document",
name="path",
field=models.CharField(db_collation="C", max_length=252, unique=True),
),
]

View File

@@ -4,20 +4,27 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0015_set_path_on_existing_documents'),
("core", "0015_set_path_on_existing_documents"),
]
operations = [
migrations.AddField(
model_name='document',
name='excerpt',
field=models.TextField(blank=True, max_length=300, null=True, verbose_name='excerpt'),
model_name="document",
name="excerpt",
field=models.TextField(
blank=True, max_length=300, null=True, verbose_name="excerpt"
),
),
migrations.AlterField(
model_name='user',
name='language',
field=models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language'),
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
]

View File

@@ -4,33 +4,49 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0016_add_document_excerpt'),
("core", "0016_add_document_excerpt"),
]
operations = [
migrations.AlterModelOptions(
name='document',
options={'ordering': ('path',), 'verbose_name': 'Document', 'verbose_name_plural': 'Documents'},
name="document",
options={
"ordering": ("path",),
"verbose_name": "Document",
"verbose_name_plural": "Documents",
},
),
migrations.AddField(
model_name='document',
name='ancestors_deleted_at',
model_name="document",
name="ancestors_deleted_at",
field=models.DateTimeField(blank=True, null=True),
),
migrations.AddField(
model_name='document',
name='deleted_at',
model_name="document",
name="deleted_at",
field=models.DateTimeField(blank=True, null=True),
),
migrations.AlterField(
model_name='user',
name='language',
field=models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language'),
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
migrations.AddConstraint(
model_name='document',
constraint=models.CheckConstraint(condition=models.Q(('deleted_at__isnull', True), ('deleted_at', models.F('ancestors_deleted_at')), _connector='OR'), name='check_deleted_at_matches_ancestors_deleted_at_when_set'),
model_name="document",
constraint=models.CheckConstraint(
condition=models.Q(
("deleted_at__isnull", True),
("deleted_at", models.F("ancestors_deleted_at")),
_connector="OR",
),
name="check_deleted_at_matches_ancestors_deleted_at_when_set",
),
),
]

View File

@@ -0,0 +1,24 @@
from django.db import migrations
def update_titles_to_null(apps, schema_editor):
"""
If the titles are "Untitled document" or "Unbenanntes Dokument" or "Document sans titre"
we set them to Null
"""
Document = apps.get_model("core", "Document")
Document.objects.filter(
title__in=["Untitled document", "Unbenanntes Dokument", "Document sans titre"]
).update(title=None)
class Migration(migrations.Migration):
dependencies = [
("core", "0017_add_fields_for_soft_delete"),
]
operations = [
migrations.RunPython(
update_titles_to_null, reverse_code=migrations.RunPython.noop
),
]

View File

@@ -0,0 +1,36 @@
# Generated by Django 5.1.5 on 2025-03-04 12:23
from django.db import migrations, models
import core.models
class Migration(migrations.Migration):
dependencies = [
("core", "0018_update_blank_title"),
]
operations = [
migrations.AlterModelManagers(
name="user",
managers=[
("objects", core.models.UserManager()),
],
),
migrations.AlterField(
model_name="user",
name="language",
field=models.CharField(
blank=True,
choices=[
("en-us", "English"),
("fr-fr", "Français"),
("de-de", "Deutsch"),
],
default=None,
help_text="The language in which the user wants to see the interface.",
max_length=10,
null=True,
verbose_name="language",
),
),
]

View File

@@ -0,0 +1,77 @@
# Generated by Django 5.1.4 on 2025-01-18 11:53
import re
import django.contrib.postgres.fields
import django.db.models.deletion
from django.core.files.storage import default_storage
from django.db import migrations, models
from botocore.exceptions import ClientError
import core.models
from core.utils import extract_attachments
def populate_attachments_on_all_documents(apps, schema_editor):
"""Populate "attachments" field on all existing documents in the database."""
Document = apps.get_model("core", "Document")
for document in Document.objects.all():
try:
response = default_storage.connection.meta.client.get_object(
Bucket=default_storage.bucket_name, Key=f"{document.pk!s}/file"
)
except (FileNotFoundError, ClientError):
pass
else:
content = response["Body"].read().decode("utf-8")
document.attachments = extract_attachments(content)
document.save(update_fields=["attachments"])
class Migration(migrations.Migration):
dependencies = [
("core", "0019_alter_user_language_default_to_null"),
]
operations = [
# v2.0.0 was released so we can now remove BC field "is_public"
migrations.RemoveField(
model_name="document",
name="is_public",
),
migrations.AlterModelManagers(
name="user",
managers=[
("objects", core.models.UserManager()),
],
),
migrations.AddField(
model_name="document",
name="attachments",
field=django.contrib.postgres.fields.ArrayField(
base_field=models.CharField(max_length=255),
blank=True,
default=list,
editable=False,
null=True,
size=None,
),
),
migrations.AddField(
model_name="document",
name="duplicated_from",
field=models.ForeignKey(
blank=True,
editable=False,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="duplicates",
to="core.document",
),
),
migrations.RunPython(
populate_attachments_on_all_documents,
reverse_code=migrations.RunPython.noop,
),
]

View File

@@ -0,0 +1,10 @@
from django.contrib.postgres.operations import UnaccentExtension
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
("core", "0020_remove_is_public_add_field_attachments_and_duplicated_from"),
]
operations = [UnaccentExtension()]

View File

@@ -0,0 +1,89 @@
# Generated by Django 5.2.3 on 2025-06-18 10:02
import uuid
import django.db.models.deletion
from django.conf import settings
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("core", "0021_activate_unaccent_extension"),
]
operations = [
migrations.CreateModel(
name="DocumentAskForAccess",
fields=[
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
(
"role",
models.CharField(
choices=[
("reader", "Reader"),
("editor", "Editor"),
("administrator", "Administrator"),
("owner", "Owner"),
],
default="reader",
max_length=20,
),
),
(
"document",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="ask_for_accesses",
to="core.document",
),
),
(
"user",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="ask_for_accesses",
to=settings.AUTH_USER_MODEL,
),
),
],
options={
"verbose_name": "Document ask for access",
"verbose_name_plural": "Document ask for accesses",
"db_table": "impress_document_ask_for_access",
"constraints": [
models.UniqueConstraint(
fields=("user", "document"),
name="unique_document_ask_for_access_user",
violation_error_message="This user has already asked for access to this document.",
)
],
},
),
]

View File

@@ -0,0 +1,17 @@
# Generated by Django 5.1.7 on 2025-03-14 14:03
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("core", "0022_alter_user_language_documentaskforaccess"),
]
operations = [
migrations.AddField(
model_name="document",
name="has_deleted_children",
field=models.BooleanField(default=False),
),
]

Some files were not shown because too many files have changed in this diff Show More