Compare commits

...

68 Commits

Author SHA1 Message Date
virgile-deville
c924706aa8 📝(doc) change documentation folder name
It was called docs which is confusing as it contain the code of the app

Signed-off-by: virgile-deville <virgile.deville@beta.gouv.fr>
2025-04-17 10:44:54 +02:00
Anthony LC
ecd06560c6 🚚(frontend) Display homepage on /home url
The homepage is now accessible at the /home URL.
Before the homepage was accessible on the /login URL.
We still keep the /login URL for backward compatibility.
2025-04-13 13:25:40 +02:00
Anthony LC
e9ab099ce0 🚩(frontend) integrate homepage feature flag
If the homepage feature flag is enabled,
the homepage will be displayed.
2025-04-13 13:25:40 +02:00
Anthony LC
67b69d05f7 🚩(backend) add homepage feature flag
Add a homepage feature flag that we will
propagate to the frontend.
It will be used to enable or disable the
homepage at runtime.
2025-04-13 13:25:40 +02:00
virgile-deville
f429eb053a 📝(readme) remove preprod account
Having a preprod account using a yopmail account was a security bad practice

Co-authored-by: Samuel Paccoud <sampaccoud@users.noreply.github.com>
2025-04-11 22:29:22 +02:00
Olivier Laurendeau
ad11b7f554 📝(docs) init architecture documentation
- Add docs about architecture
- Add ADR about the CRDT choice
2025-04-10 15:15:15 +02:00
Anthony LC
3d5adad227 🔖(minor) release 3.1.0
Added:
- 🚩(backend) add feature flag for the footer
- 🔧(backend) add view to manage footer json
- (frontend) add custom css style
- 🚩(frontend) conditionally render AI button only
  when feature is enabled

Changed:
- 🚨(frontend) block button when creating doc

Fixed:
- 🐛(back) validate document content in serializer
- 🐛(frontend) fix selection click
  past end of content
2025-04-08 12:41:38 +02:00
Anthony LC
de8e812f2f 🥚(frontend) remove easter egg
Remove the easter egg from the console.
2025-04-07 13:18:04 +02:00
Nathan Panchout
7a1601c682 (frontend) update favicon files and links
- Added new favicon files: favicon-dark.png and favicon.png.
- Updated the _app.tsx file to link to the new favicon files, supporting
both light and dark color schemes.
2025-04-07 13:18:04 +02:00
Nathan Panchout
0537572542 ♻️(frontend) icon component refactoring
- Add variant to IconComponent and remove $isMaterialIcon prop
- Replace all Text component used as icon with the Icon component.
2025-04-07 13:18:04 +02:00
Anthony LC
8aab007ad1 🐛(frontend) do not display firefox modal if not necessary
It is necessary to display the firefox modal only
if the user has something to save.
2025-04-07 13:18:04 +02:00
Anthony LC
cde3de43f7 ♻️(frontend) misc improvements
- add opacity props on Box
- rename to cunningham-style.css
- update illustration-docs-empty.png
- smaller tooltip
2025-04-07 13:18:04 +02:00
Anthony LC
8c0c3c2f44 🐛(frontend) fix selection click past end of content
On Chrome, when we click at the end of a line,
the cursor is placed at the beginning of the line.
We fix this behavior, now the cursor is placed
at the end of the line.
2025-04-07 13:18:04 +02:00
Anthony LC
c11d59c434 🚩(backend) add feature flag for the footer
We added the feature flag `FRONTEND_FOOTER_FEATURE_ENABLED`
to enable or disable the footer in the frontend.
2025-04-04 15:44:38 +02:00
Manuel Raynaud
8836109945 ♻️(back) reset cache after every test
We move the cache reset in the global conf test to not have to think
about reseting the cache when we implement test.
2025-04-04 15:44:38 +02:00
Anthony LC
ba136ff82f 🔧(backend) add view to manage footer json
We added the `FRONTEND_URL_JSON_FOOTER` environment
variable. It will give the possibility to generate
your own footer content in the frontend.
If the variable is not set, the footer will not
be displayed.
2025-04-04 15:44:38 +02:00
Anthony LC
96d9d1a184 🔊(y-provider) improve and add logs
We have somes entries with "No cookies", we
add more logs to understand why we have this case.
We add the datetime in front of each entries as
well.
2025-04-03 16:27:40 +02:00
Manuel Raynaud
771ffdc7cc 🔥(y-provider) remove npm in docker image
We use yarn and not npm, we remove npm because it has a dependencie with
cross-spawn which has a CVE.
2025-04-03 10:41:55 +02:00
Manuel Raynaud
82eba1e8ea 🔥(ci) force ci to fails if trivy fails
If trivy fails we must stop the CI to avoid publishing images with
security issues.
2025-04-03 10:41:55 +02:00
renovate[bot]
8c42599d0f ⬆️(dependencies) update django to v5.1.8 [SECURITY] 2025-04-03 10:28:12 +02:00
renovate[bot]
8620cf4857 ⬆️(dependencies) update next to v15.2.4 [SECURITY] 2025-04-03 07:02:20 +02:00
Baptiste Fontaine
2a7da73248 📝(docs) fix grammar and typos
Fix grammar and typos in docs/installation.md file.
2025-04-02 11:53:56 +02:00
Manuel Raynaud
e8e9922832 (back) remove url-normalize dependency
We have the dependency url-normalize installed but we don't use it in
our codebase.
2025-04-01 09:51:29 +02:00
renovate[bot]
2da4ce4570 ⬆️(dependencies) update python dependencies 2025-04-01 09:51:29 +02:00
Anthony LC
50b90f9ae7 (e2e) fix some flaky tests
Some tests were flaky, causing them to fail
intermittently. This commit aims to address
this issue.
2025-03-31 12:34:04 +02:00
Anthony LC
65ddf7fbe8 📝(docs) add documentation about runtime theming
Add a documentation page about runtime theming.
It explains how to use the theming system
and provide a example.
2025-03-31 12:34:04 +02:00
Anthony LC
d3a7ee74b3 💄(frontend) add classname to components
To be easily customized, we added a classname
to most of the components.
2025-03-31 12:34:04 +02:00
Anthony LC
65e450c6cc (frontend) add custom css style
From the config, we can add custom css style
to the app.
2025-03-31 12:34:04 +02:00
Anthony LC
725cae5470 🔧(backend) add FRONTEND_CSS_URL env var
We added the `FRONTEND_CSS_URL` environment
variable. It will give the possibility to add a
css layer at runtime.
2025-03-31 12:34:04 +02:00
Anthony LC
3881930e82 🚨(frontend) block button when creating doc
When the user clicks on the button to create a new doc,
the button is disabled to prevent multiple clicks.
Multiple clicks on the button could create multiple docs
and create a error about duplicated paths.
2025-03-31 11:29:28 +02:00
Anthony LC
910686293c ️(frontend) improve heading store
Reset headings only when the headers are not
equal to the previous ones. It will prevent
unnecessary rerenders.
2025-03-31 11:29:28 +02:00
Anthony LC
7e7c9ac4c5 ♻️(frontend) replace useModal hook
useModal hook does not use useCallback for its
methods that creates useless rerenders.
2025-03-31 11:29:28 +02:00
Anthony LC
d5d2cfab8e ♻️(frontend) improve useSaveDoc hook
- add tests for useSaveDoc hook
- simplify hooks states
- reduce useEffect calls
2025-03-31 11:29:28 +02:00
Matthias
f2ed8e0ea1 🐛(frontend) conditionally render AI button in toolbar
Added a feature flag check to ensure the AIGroupButton is only rendered
when AI_FEATURE_ENABLED is explicitly set to "true". This prevents the
AI button from appearing when the feature is not configured or disabled.

Fixes #782

Signed-off-by: Matthias <matthias@universum.com>
2025-03-31 11:04:00 +02:00
Manuel Raynaud
fbe8a26dba 🐛(back) validate document content in serializer
We recently extract images url in the content. For this, we assume that
the document content is always in base64. We enforce this assumption by
checking if it's a valide base64 in the serializer.
2025-03-29 19:08:39 +01:00
Berry den Hartog
3e974be9f4 📝(docs) describe environmental options for docs backend (#821)
Signed-off-by: 
Berry den Hartog <38954346+berrydenhartog@users.noreply.github.com>
2025-03-28 17:15:35 +00:00
Bastien Guerry
10f9d25920 📄(legal) add warning about legal compliance
We need to double-check our legal constraints regarding the use of XL
packages within Docs. In the meantime, sends a message to potential
reusers.
2025-03-28 17:15:16 +01:00
Jacques ROUSSEL
4178693e63 🐛(ci) use github action for argocd webhook notification
In order to refactor this notification between alls projetcs, we choose
to use a custom github action
2025-03-28 16:42:45 +01:00
Anthony LC
53be6de5f8 🔖(major) release 3.0.0
Added:
- 📄(legal) Require contributors to sign a DCO

Changed:
- ♻️(frontend) Integrate UI kit
- 🏗️(y-provider) manage auth in y-provider app

Fixed:
- 🐛(backend) compute ancestor_links in get_abilities
  if needed
- 🔒️(back) restrict access to document accesses
2025-03-28 15:32:08 +01:00
Anthony LC
4ff90abdee 🐛(service-worker) force reload new service worker
When multiple tabs are open, the new service worker
can stay in the "waiting" state and not be activated
until the other tabs with the old service worker
are closed.
We fix this by forcing the other tabs to reload
the page when a new service worker is detected.
All tabs will then be reloaded and the new service
worker will be activated.
2025-03-28 15:32:08 +01:00
Anthony LC
544dd00c16 🔧(helm) adapt setting helm dev file
The way that collaboration server authentifies the user
has changed. We adapt the configuration to the new
way of doing it, by removing the nginx auth url,
and by adding COLLABORATION_BACKEND_BASE_URL
setting.
2025-03-28 15:32:08 +01:00
Anthony LC
a3cd4c51ea 🩹(frontend) fine tunning for v3.0.0
- fix width select export
2025-03-28 15:32:08 +01:00
Manuel Raynaud
7e1eed3abd (y-provider) check hocuspocus documentName validity
We only use uuid v4 as hocuspocus dicument name. To be sure nothing else
is used we check that the documentName is a valid uuid version 4.
2025-03-27 18:42:04 +01:00
Manuel Raynaud
8bee476b5b 🔥(back) remove collaboration-auth endpoint
We don't need anymore the collaboration-auth endpoint. Every code
related to it is removed.
2025-03-27 18:42:04 +01:00
Manuel Raynaud
e86919fb9a 🏗️(y-provider) manage auth in y-provider app
The way to connect to the hocuspocus server needs to be proxified in
nginx to query a dedicated route in the django application and then
follow the request to the express server with the additionnal headers.
The auth can be done in the express server by querying the backend on
the document retrieve endpoint. If the response status code is 200, the
user has access to the document, otherwise it is not the case. Then we
can check the abilities to determine what the user can do or not.
2025-03-27 18:42:04 +01:00
Manuel Raynaud
a5b9169eb6 ♻️(back) replace Ypy by pycrdt
Ypy is deprecated and unmaintained. We have problem with parsing
existing documents. We replace it by pycrdt, library actively maintained
and without the issues we have with Ypy.
2025-03-27 18:27:04 +01:00
Manuel Raynaud
c0dfb4b6b3 ♻️(back) remove filtering on logging handler
Level filtering was used on the logging console handler. We remove as it
is not necessary to have it.
2025-03-27 18:27:04 +01:00
Manuel Raynaud
be051ad7d2 🐛(ci) use sha256 to sign argocd webhook call
The argocd webhook call needs now to use sha256 digest now to sign
2025-03-27 18:27:04 +01:00
Manuel Raynaud
a4452784e1 🔒️(back) restrict accesss to document accesses
Every user having an access to a document, no matter its role have
access to the entire accesses list with all the user details. Only
owner or admin should be able to have the entire list, for the other
roles, they have access to the list containing only owner and
administrator with less information on the username. The email and its
id is removed
2025-03-26 10:40:53 +01:00
Quentin BEY
2929e98260 ♻️(documents) inherit manager from queryset
During a code review, I saw we are overriding the MP_NodeManager and
redefine the queryset filters:

- The MP_NodeManager sorts the queryset by `path` by default and it's
  not done on our side, is it on purpose?
- The fact we need to redefine `readable_per_se` as a boilerplate is
  surprising.

I suggest we use the Django mechanism to generate the manager from the
queryset.
2025-03-24 15:04:50 +01:00
Manuel Raynaud
a1914c6259 🐛(backend) compute ancestor_links in get_abilities if needed
The refactor made in the tree view caching the ancestors_links to not
compute them again in the document.get_abilities method lead to a bug.
If the get_abilities method is called without ancestors_links, then they
are computed on all the ancestors but not from the highest readable
ancestor for the current user. We have to compute them with this
constraint.
2025-03-24 14:04:46 +01:00
Samuel Paccoud - DINUM
c882f1386c ♻️(backend) remove lazy from languages field on User model
The idea behind wrapping choices in `lazy` function was to allow
overriding the list of languages in tests with `override_settings`.
This was causin makemigrations to keep on including the field in
migrations when it is not needed. Since we finally don't override
the LANGUAGES setting in tests, we can remove it to fix the problem.
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
c02f19a2cd (backend) extract attachment keys from updated content for access
We can't prevent document editors from copy/pasting content to from one
document to another. The problem is that copying content, will copy the
urls pointing to attachments but if we don't do anything, the reader of
the document to which the content is being pasted, may not be allowed to
access the attachment files from the original document.

Using the work from the previous commit, we can grant access to the readers
of the target document by extracting the attachment keys from the content and
adding themto the target document's "attachments" field. Before doing this,
we check that the current user can indeed access the attachment files extracted
from the content and that they are allowed to edit the current document.
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
34a208a80d (backend) add duplicate action to the document API endpoint
We took this opportunity to refactor the way access is controlled on
media attachments. We now add the media key to a list on the document
instance each time a media is uploaded to a document. This list is
passed along when a document is duplicated, allowing us to grant
access to readers on the new document, even if they don't have or
lost access to the original document.

We also propose an option to reproduce the same access rights on the
duplicate document as what was in place on the original document.
This can be requested by passing the "with_accesses=true" option in
the query string.

The tricky point is that we need to extract attachment keys from the
existing documents and set them on the new "attachments" field that is
now used to track access rights on media files.
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
6976bb7c78 (backend) fix migration test using model factory
Migration tests should not import and use factories or models
directly from the code because they would not be in sync with
the database in the state that each state needs to test it.

Instead the migrator object passed as argument allows us to
retrieve a minimal version of the models in sync with the state
of the database that we are testing. What we get is a minimal
model and we need to simulate all the methods that we could have
on the real model and that are needed for testing.
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
621393165f (backend) add missing test on media-auth and collaboration-auth
These methods were involved in a bug that was fixed without first
evidencing the error in a test:
https://github.com/suitenumerique/docs/pull/556

Fixes https://github.com/suitenumerique/docs/issues/567
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
3e9b530985 (backend) add missing tests for collaboration auth
Tests were forgotten. While writing the tests, I fixed
a few edge cases like the possibility to connect to the
collaboration server for an anonymous user.
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
54f9b3963e ♻️(backend) refactor media_auth and collaboration_auth for flexibility
These 2 actions had factorized code but a few iterations lead to
spaghetti code where factorized code includes "if" clauses.

Refactor abstractions so that code factorization really works.
2025-03-24 10:43:45 +01:00
Samuel Paccoud - DINUM
710bbf512c (backend) add util to extract text from Ydoc content
Documents content is stored in the Ydoc format. We need a util
to extract it as xml/text.
2025-03-24 10:43:45 +01:00
Jacques ROUSSEL
747ca70186 🐛(ci) fix Tilt resources dependencies
The Tilt stack was not starting properly due to dependency issues. We
need to wait for PostgreSQL to be running before starting the migration.
2025-03-24 09:33:15 +01:00
renovate[bot]
9374495fda ⬆️(dependencies) update next to v15.2.3 [SECURITY] 2025-03-24 09:18:33 +01:00
Bastien Guerry
ef7cc67387 📄(legal) Require contributors to sign a DCO
Contributors are required to sign off their commits: this confirms
that they have read and accepted https://developercertificate.org.
2025-03-23 09:57:35 +01:00
Sylvain Zimmer
a8529e434a 🐛(media) fix compatibility with Scaleway Object Storage
Some providers with S3-compatible APIs have slightly different
implementations. In this case, Scaleway didn't accept version_id=""
and has a different version ID scheme. This was tested successfully
and should remain compatible with any other provider.
2025-03-22 18:00:43 +01:00
Manuel Raynaud
f8203a1766 🚨(back) lint code with ruff 0.11.2
New Ruff rule (C420) detects code that should be linted. We apply this
new rule on our code.
2025-03-22 10:28:48 +01:00
renovate[bot]
ce8b98e256 ⬆️(dependencies) update python dependencies 2025-03-22 10:28:48 +01:00
Anthony LC
4243519eee 🔥(frontend) remove Marianne font
Marianne font is now part of the UI kit.
We can remove it from the project.
2025-03-21 17:49:06 +01:00
Nathan Panchout
1abf529891 (frontend) refactor and theme token update
The configuration file has been simplified by importing configurations
from @gouvfr-lasuite/ui-kit . Colors and components have been updated to
reflect the new values. Additionally, adjustments have been made to
global styles, including the addition of styles for Material icons. Form
components have also been modified to incorporate the new style
properties.
2025-03-21 17:49:06 +01:00
Nathan Panchout
69ca4af539 (frontend) updated dependencies and added new packages
Added several new dependencies to the `package.json` file, including
`@dnd-kit/core`, `@dnd-kit/modifiers`, `@fontsource/material-icons`, and
`@gouvfr-lasuite/ui-kit`.
2025-03-21 17:49:06 +01:00
214 changed files with 4743 additions and 3060 deletions

View File

@@ -11,6 +11,7 @@ on:
pull_request:
branches:
- 'main'
- 'ci/trivy-fails'
env:
DOCKER_USER: 1001:127
@@ -38,7 +39,6 @@ jobs:
with:
docker-build-args: '--target backend-production -f Dockerfile'
docker-image-name: 'docker.io/lasuite/impress-backend:${{ github.sha }}'
continue-on-error: true
-
name: Build and push
uses: docker/build-push-action@v6
@@ -72,7 +72,6 @@ jobs:
with:
docker-build-args: '-f src/frontend/Dockerfile --target frontend-production'
docker-image-name: 'docker.io/lasuite/impress-frontend:${{ github.sha }}'
continue-on-error: true
-
name: Build and push
uses: docker/build-push-action@v6
@@ -106,8 +105,7 @@ jobs:
uses: numerique-gouv/action-trivy-cache@main
with:
docker-build-args: '-f src/frontend/servers/y-provider/Dockerfile --target y-provider'
docker-image-name: 'docker.io/lasuite/impress-frontend:${{ github.sha }}'
continue-on-error: true
docker-image-name: 'docker.io/lasuite/impress-y-provider:${{ github.sha }}'
-
name: Build and push
uses: docker/build-push-action@v6
@@ -127,12 +125,9 @@ jobs:
runs-on: ubuntu-latest
if: github.event_name != 'pull_request'
steps:
-
name: Checkout repository
uses: actions/checkout@v4
-
name: Call argocd github webhook
run: |
data='{"ref": "'$GITHUB_REF'","repository": {"html_url":"'$GITHUB_SERVER_URL'/${{ secrets.DEPLOYMENT_REPO_URL }}"}}'
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac "${{ secrets.ARGOCD_PREPROD_WEBHOOK_SECRET }}" | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" ${{ vars.ARGOCD_PREPROD_WEBHOOK_URL }}
- uses: numerique-gouv/action-argocd-webhook-notification@main
id: notify
with:
deployment_repo_path: "${{ secrets.DEPLOYMENT_REPO_URL }}"
argocd_webhook_secret: "${{ secrets.ARGOCD_PREPROD_WEBHOOK_SECRET }}"
argocd_url: "${{ vars.ARGOCD_PREPROD_WEBHOOK_URL }}"

View File

@@ -7,6 +7,47 @@ and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
- 📝(doc) change documentation folder name to "documentation" #876
## Added
- 🚩 add homepage feature flag #861
## [3.1.0] - 2025-04-07
## Added
- 🚩(backend) add feature flag for the footer #841
- 🔧(backend) add view to manage footer json #841
- ✨(frontend) add custom css style #771
- 🚩(frontend) conditionally render AI button only when feature is enabled #814
## Changed
- 🚨(frontend) block button when creating doc #749
## Fixed
- 🐛(back) validate document content in serializer #822
- 🐛(frontend) fix selection click past end of content #840
## [3.0.0] - 2025-03-28
## Added
- 📄(legal) Require contributors to sign a DCO #779
## Changed
- ♻️(frontend) Integrate UI kit #783
- 🏗️(y-provider) manage auth in y-provider app #804
## Fixed
- 🐛(backend) compute ancestor_links in get_abilities if needed #725
- 🔒️(back) restrict access to document accesses #801
## [2.6.0] - 2025-03-21
@@ -124,6 +165,8 @@ and this project adheres to
## Added
- ✨(backend) add duplicate action to the document API endpoint
- ⚗️(backend) add util to extract text from base64 yjs document
- ✨(backend) add soft delete and restore API endpoints to documents #516
- ✨(backend) allow organizing documents in a tree structure #516
- ✨(backend) add "excerpt" field to document list serializer #516
@@ -487,8 +530,10 @@ and this project adheres to
- ✨(frontend) Coming Soon page (#67)
- 🚀 Impress, project to manage your documents easily and collaboratively.
[unreleased]: https://github.com/numerique-gouv/impress/compare/v2.6.0...main
[v2.5.0]: https://github.com/numerique-gouv/impress/releases/v2.6.0
[unreleased]: https://github.com/numerique-gouv/impress/compare/v3.1.0...main
[v3.1.0]: https://github.com/numerique-gouv/impress/releases/v3.1.0
[v3.0.0]: https://github.com/numerique-gouv/impress/releases/v3.0.0
[v2.6.0]: https://github.com/numerique-gouv/impress/releases/v2.6.0
[v2.5.0]: https://github.com/numerique-gouv/impress/releases/v2.5.0
[v2.4.0]: https://github.com/numerique-gouv/impress/releases/v2.4.0
[v2.3.0]: https://github.com/numerique-gouv/impress/releases/v2.3.0

View File

@@ -4,6 +4,8 @@ Thank you for taking the time to contribute! Please follow these guidelines to e
To get started with the project, please refer to the [README.md](https://github.com/suitenumerique/docs/blob/main/README.md) for detailed instructions.
Contributors are required to sign off their commits with `git commit --sign-off`: this confirms that they have read and accepted the [Developer's Certificate of Origin 1.1](https://developercertificate.org/).
Please also check out our [dev handbook](https://suitenumerique.gitbook.io/handbook) to learn our best practices.
## Help us with translations

View File

@@ -15,6 +15,13 @@ FROM base AS back-builder
WORKDIR /builder
# Install Rust and Cargo using Alpine's package manager
RUN apk add --no-cache \
build-base \
libffi-dev \
rust \
cargo
# Copy required python dependencies
COPY ./src/backend /builder

View File

@@ -1,6 +1,6 @@
<p align="center">
<a href="https://github.com/suitenumerique/docs">
<img alt="Docs" src="/docs/assets/docs-logo.png" width="300" />
<img alt="Docs" src="/documentation/assets/docs-logo-black-white.png" width="300" />
</a>
</p>
@@ -20,10 +20,12 @@ Welcome to Docs! The open source document editor where your notes can become kno
</a>
</p>
<img src="/docs/assets/docs_live_collaboration_light.gif" width="100%" align="center"/>
<img src="/documentation/assets/docs_live_collaboration_light.gif" width="100%" align="center"/>
## Why use Docs ❓
⚠️ **Note that Docs provides docs/pdf exporters by loading [two BlockNote packages](https://github.com/suitenumerique/docs/blob/main/src/frontend/apps/impress/package.json#L22C7-L23C53), which we use under the AGPL-3.0 licence. Until we comply with the terms of this license, we recommend that you don't run Docs as a commercial product, unless you are willing to sponsor [BlockNote](https://github.com/TypeCellOS/BlockNote).**
Docs is a collaborative text editor designed to address common challenges in knowledge building and sharing.
### Write
@@ -46,12 +48,7 @@ Docs is a collaborative text editor designed to address common challenges in kno
### Test it
Test Docs on your browser by logging in on this [environment](https://impress-preprod.beta.numerique.gouv.fr/)
```
email: test.docs@yopmail.com
password: I'd<3ToTestDocs
```
Test Docs on your browser by visiting this [demo document](https://impress-preprod.beta.numerique.gouv.fr/docs/6ee5aac4-4fb9-457d-95bf-bb56c2467713/)
### Run it locally
@@ -196,5 +193,5 @@ We are proud sponsors of [BlockNotejs](https://www.blocknotejs.org/) and [Yjs](h
We are always looking for new public partners (we are currently onboarding the Netherlands 🇳🇱🧀), feel free to [reach out](mailto:docs@numerique.gouv.fr) if you are interested in using or contributing to Docs.
<p align="center">
<img src="/docs/assets/europe_opensource.png" width="50%"/>
<img src="/documentation/assets/europe_opensource.png" width="50%"/>
</p>

View File

@@ -16,6 +16,18 @@ the following command inside your docker container:
## [Unreleased]
## [3.0.0] - 2025-03-28
We are not using the nginx auth request anymore to access the collaboration server (`yProvider`)
The authentication is now managed directly from the yProvider server.
You must remove the annotation `nginx.ingress.kubernetes.io/auth-url` from the `ingressCollaborationWS`.
This means as well that the yProvider server must be able to access the Django server.
To do so, you must set the `COLLABORATION_BACKEND_BASE_URL` environment variable to the `yProvider`
service.
## [2.2.0] - 2025-02-10
- AI features are now limited to users who are authenticated. Before this release, even anonymous
users who gained editor access on a document with link reach used to get AI feature.
IF you want anonymous users to keep access on AI features, you must now define the

View File

@@ -39,6 +39,9 @@ docker_build(
]
)
k8s_resource('impress-docs-backend-migrate', resource_deps=['postgres-postgresql'])
k8s_resource('impress-docs-backend-createsuperuser', resource_deps=['impress-docs-backend-migrate'])
k8s_resource('impress-docs-backend', resource_deps=['impress-docs-backend-migrate'])
k8s_yaml(local('cd ../src/helm && helmfile -n impress -e dev template .'))
migration = '''

View File

@@ -185,11 +185,15 @@ services:
context: .
dockerfile: ./src/frontend/servers/y-provider/Dockerfile
target: y-provider
command: ["yarn", "workspace", "server-y-provider", "run", "dev"]
working_dir: /app/frontend
restart: unless-stopped
env_file:
- env.d/development/common
ports:
- "4444:4444"
volumes:
- ./src/frontend/:/app/frontend
kc_postgresql:
image: postgres:14.3

View File

@@ -4,54 +4,6 @@ server {
server_name localhost;
charset utf-8;
# Proxy auth for collaboration server
location /collaboration/ws/ {
# Collaboration Auth request configuration
auth_request /collaboration-auth;
auth_request_set $authHeader $upstream_http_authorization;
auth_request_set $canEdit $upstream_http_x_can_edit;
auth_request_set $userId $upstream_http_x_user_id;
# Pass specific headers from the auth response
proxy_set_header Authorization $authHeader;
proxy_set_header X-Can-Edit $canEdit;
proxy_set_header X-User-Id $userId;
# Ensure WebSocket upgrade
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
# Collaboration server
proxy_pass http://y-provider:4444;
# Set appropriate timeout for WebSocket
proxy_read_timeout 86400;
proxy_send_timeout 86400;
# Preserve original host and additional headers
proxy_set_header Host $host;
}
location /collaboration-auth {
proxy_pass http://app-dev:8000/api/v1.0/documents/collaboration-auth/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Original-URL $request_uri;
# Prevent the body from being passed
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-Method $request_method;
}
location /collaboration/api/ {
# Collaboration server
proxy_pass http://y-provider:4444;
proxy_set_header Host $host;
}
# Proxy auth for media
location /media/ {
# Auth request configuration

Binary file not shown.

Before

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.3 KiB

View File

@@ -0,0 +1,193 @@
## Decision TLDR;
We will use Yjs a CRDT-based library for the collaborative editing of the documents.
## Status
Accepted
## Context
We need to implement a collaborative editing feature for the documents that supports real-time collaboration, offline capabilities, and seamless integration with our Django backend.
## Considered alternatives
### ProseMirror
A robust toolkit for building rich-text editors with collaboration capabilities.
| Pros | Cons |
| --- | --- |
| Mature ecosystem | Complex integration with Django |
| Rich text editing features | Steeper learning curve |
| Used by major companies | More complex to implement offline support |
| Large community | |
### ShareDB
Real-time database backend based on Operational Transformation.
| Pros | Cons |
| --- | --- |
| Battle-tested in production | Complex setup required |
| Strong consistency model | Requires specific backend architecture |
| Good documentation | Less flexible with different backends |
| | Higher latency compared to CRDTs |
### Convergence
Complete enterprise solution for real-time collaboration.
| Pros | Cons |
| --- | --- |
| Full-featured solution | Commercial licensing |
| Built-in presence features | Less community support |
| Enterprise support | More expensive |
| Good offline support | Overkill for basic needs |
### CRDT-based Solutions Comparison
A CRDT-based library specifically designed for real-time collaboration.
| Category | Pros | Cons |
|----------|------|------|
| Technical Implementation | • Native real-time collaboration<br>• No central conflict resolution needed<br>• Works well with Django backend<br>• Automatic state synchronization | • Learning curve for CRDT concepts<br>• More complex initial setup<br>• Additional metadata overhead |
| User Experience | • Instant local updates<br>• Works offline by default<br>• Low latency<br>• Smooth concurrent editing | • Eventual consistency might cause brief inconsistencies<br>• UI must handle temporary conflicts |
| Performance | • Excellent scaling with multiple users<br>• Reduced server load<br>• Efficient network usage<br>• Good memory optimization (especially Yjs) | • Slightly higher memory usage<br>• Initial state sync can be larger |
| Development | • No need to build conflict resolution<br>• Simple integration with text editors<br>• Future-proof architecture | • Team needs to learn new concepts<br>• Fewer ready-made solutions<br>• May need to build some features from scratch |
| Maintenance | • Less server infrastructure<br>• Simpler deployment<br>• Fewer points of failure | • Debugging can be more complex<br>• State management requires careful handling |
| Business Impact | • Better offline support for users<br>• Scales well as user base grows<br>• No licensing costs (with Yjs) | • Initial development time might be longer<br>• Team training required |
#### Yjs
- **Type**: State-based CRDT
- **Implementation**: JavaScript/TypeScript
- **Features**:
- Rich text collaboration
- Shared types (Array, Map, XML)
- Binary encoding
- P2P support
- **Performance**: Excellent for text editing
- **Memory Usage**: Optimized
- **License**: MIT
#### Automerge
- **Type**: Operation-based CRDT
- **Implementation**: JavaScript/Rust
- **Features**:
- JSON-like data structures
- Change history
- Undo/Redo
- Binary format
- **Performance**: Good, with Rust backend
- **Memory Usage**: Higher than Yjs
- **License**: MIT
#### Legion
- **Type**: State-based CRDT
- **Implementation**: Rust with JS bindings
- **Features**:
- High performance
- Memory efficient
- Binary protocol
- **Performance**: Excellent
- **Memory Usage**: Very efficient
- **License**: Apache 2.0
#### Diamond Types
- **Type**: Operation-based CRDT
- **Implementation**: TypeScript
- **Features**:
- Specialized for text
- Small memory footprint
- Simple API
- **Performance**: Good for text
- **Memory Usage**: Efficient
- **License**: MIT
Comparison Table:
| Feature | Yjs | Automerge | Legion | Diamond Types |
|---------|-----|-----------|--------|---------------|
| Text Editing | ✅ Excellent | ✅ Good | ⚠️ Basic | ✅ Excellent |
| Structured Data | ✅ | ✅ | ✅ | ⚠️ |
| Memory Efficiency | ✅ High | ⚠️ Medium | ✅ Very High | ✅ High |
| Network Efficiency | ✅ | ⚠️ | ✅ | ✅ |
| Maturity | ✅ | ✅ | ⚠️ | ⚠️ |
| Community Size | ✅ Large | ✅ Large | ⚠️ Small | ⚠️ Small |
| Documentation | ✅ | ✅ | ⚠️ | ⚠️ |
| Backend Options | ✅ Many | ✅ Many | ⚠️ Limited | ⚠️ Limited |
Key Differences:
1. **Implementation Approach**:
- Yjs: Optimized for text and rich-text editing
- Automerge: General-purpose JSON CRDT
- Legion: Performance-focused with Rust
- Diamond Types: Specialized for text collaboration
2. **Performance Characteristics**:
- Yjs: Best for text editing scenarios
- Automerge: Good all-around performance
- Legion: Excellent raw performance
- Diamond Types: Optimized for text
3. **Ecosystem Integration**:
- Yjs: Wide range of integrations
- Automerge: Good JavaScript ecosystem
- Legion: Limited but growing
- Diamond Types: Focused on text editors
This analysis reinforces our choice of Yjs for the CRDT-based option as it provides:
- Best-in-class text editing performance
- Mature ecosystem
- Active community
- Excellent documentation
- Wide range of backend options
## Decision
After evaluating the alternatives, we choose Yjs for the following reasons:
1. **Technical Fit:**
- Native CRDT support ensures reliable collaboration
- Excellent offline capabilities
- Good performance characteristics
- Flexible backend integration options
2. **Project Requirements Match:**
- Easy integration with our Django backend
- Supports our core collaborative features
- Manageable learning curve for the team
3. **Community & Support:**
- Active development
- Growing community
- Good documentation
- Open source with MIT license
### Comparison of Key Features:
| Feature | Yjs (CRDT) | ProseMirror | ShareDB | Convergence |
|---------|-----|-------------|----------|-------------|
| Real-time Collaboration | ✅ | ✅ | ✅ | ✅ |
| Offline Support | ✅ | ⚠️ | ⚠️ | ✅ |
| Django Integration | Easy | Complex | Complex | Moderate |
| Learning Curve | Medium | High | High | Medium |
| Cost | Free | Free | Free | Paid |
| Community Size | Growing | Large | Medium | Small |
## Consequences
### Positive
- Simplified implementation of real-time collaboration
- Good developer experience
- Future-proof technology choice
- No licensing costs
### Negative
- Team needs to learn CRDT concepts
- Newer technology compared to alternatives
- May need to build some features available out-of-the-box in other solutions
### Risks
- Community support might not grow as expected
- May discover limitations as we scale

View File

@@ -0,0 +1,19 @@
## Architecture
### Global system architecture
```mermaid
flowchart TD
User -- HTTP --> Front("Frontend (NextJS SPA)")
Front -- REST API --> Back("Backend (Django)")
Front -- WebSocket --> Yserver("Microservice Yjs (Express)") -- WebSocket --> CollaborationServer("Collaboration server (Hocuspocus)") -- REST API <--> Back
Front -- OIDC --> Back -- OIDC ---> OIDC("Keycloak / ProConnect")
Back -- REST API --> Yserver
Back --> DB("Database (PostgreSQL)")
Back <--> Celery --> DB
Back ----> S3("Minio (S3)")
```
### Architecture decision records
- [ADR-0001-20250106-use-yjs-for-docs-editing](./adr/ADR-0001-20250106-use-yjs-for-docs-editing.md)

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

View File

Before

Width:  |  Height:  |  Size: 5.8 MiB

After

Width:  |  Height:  |  Size: 5.8 MiB

View File

Before

Width:  |  Height:  |  Size: 80 KiB

After

Width:  |  Height:  |  Size: 80 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.7 KiB

99
documentation/env.md Normal file
View File

@@ -0,0 +1,99 @@
# Docs variables
Here we describe all environment variables that can be set for the docs application.
## impress-backend container
These are the environmental variables you can set for the impress-backend container.
| Option | Description | default |
| ----------------------------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------- |
| DJANGO_ALLOWED_HOSTS | allowed hosts | [] |
| DJANGO_SECRET_KEY | secret key | |
| DJANGO_SERVER_TO_SERVER_API_TOKENS | | [] |
| DB_ENGINE | engine to use for database connections | django.db.backends.postgresql_psycopg2 |
| DB_NAME | name of the database | impress |
| DB_USER | user to authenticate with | dinum |
| DB_PASSWORD | password to authenticate with | pass |
| DB_HOST | host of the database | localhost |
| DB_PORT | port of the database | 5432 |
| MEDIA_BASE_URL | | |
| STORAGES_STATICFILES_BACKEND | | whitenoise.storage.CompressedManifestStaticFilesStorage |
| AWS_S3_ENDPOINT_URL | S3 endpoint | |
| AWS_S3_ACCESS_KEY_ID | access id for s3 endpoint | |
| AWS_S3_SECRET_ACCESS_KEY | access key for s3 endpoint | |
| AWS_S3_REGION_NAME | region name for s3 endpoint | |
| AWS_STORAGE_BUCKET_NAME | bucket name for s3 endpoint | impress-media-storage |
| DOCUMENT_IMAGE_MAX_SIZE | maximum size of document in bytes | 10485760 |
| LANGUAGE_CODE | default language | en-us |
| API_USERS_LIST_THROTTLE_RATE_SUSTAINED | throttle rate for api | 180/hour |
| API_USERS_LIST_THROTTLE_RATE_BURST | throttle rate for api on burst | 30/minute |
| SPECTACULAR_SETTINGS_ENABLE_DJANGO_DEPLOY_CHECK | | false |
| TRASHBIN_CUTOFF_DAYS | trashbin cutoff | 30 |
| DJANGO_EMAIL_BACKEND | email backend library | django.core.mail.backends.smtp.EmailBackend |
| DJANGO_EMAIL_BRAND_NAME | brand name for email | |
| DJANGO_EMAIL_HOST | host name of email | |
| DJANGO_EMAIL_HOST_USER | user to authenticate with on the email host | |
| DJANGO_EMAIL_HOST_PASSWORD | password to authenticate with on the email host | |
| DJANGO_EMAIL_LOGO_IMG | logo for the email | |
| DJANGO_EMAIL_PORT | port used to connect to email host | |
| DJANGO_EMAIL_USE_TLS | use tls for email host connection | false |
| DJANGO_EMAIL_USE_SSL | use sstl for email host connection | false |
| DJANGO_EMAIL_FROM | email adress used as sender | from@example.com |
| DJANGO_CORS_ALLOW_ALL_ORIGINS | allow all CORS origins | true |
| DJANGO_CORS_ALLOWED_ORIGINS | list of origins allowed for CORS | [] |
| DJANGO_CORS_ALLOWED_ORIGIN_REGEXES | list of origins allowed for CORS using regulair expressions | [] |
| SENTRY_DSN | sentry host | |
| COLLABORATION_API_URL | collaboration api host | |
| COLLABORATION_SERVER_SECRET | collaboration api secret | |
| COLLABORATION_WS_URL | collaboration websocket url | |
| FRONTEND_CSS_URL | To add a external css file to the app | |
| FRONTEND_HOMEPAGE_FEATURE_ENABLED | frontend feature flag to display the homepage | false |
| FRONTEND_FOOTER_FEATURE_ENABLED | frontend feature flag to display the footer | false |
| FRONTEND_FOOTER_VIEW_CACHE_TIMEOUT | Cache duration of the json footer | 86400 |
| FRONTEND_URL_JSON_FOOTER | Url with a json to configure the footer | |
| FRONTEND_THEME | frontend theme to use | |
| POSTHOG_KEY | posthog key for analytics | |
| CRISP_WEBSITE_ID | crisp website id for support | |
| DJANGO_CELERY_BROKER_URL | celery broker url | redis://redis:6379/0 |
| DJANGO_CELERY_BROKER_TRANSPORT_OPTIONS | celery broker transport options | {} |
| OIDC_CREATE_USER | create used on OIDC | false |
| OIDC_RP_SIGN_ALGO | verification algorithm used OIDC tokens | RS256 |
| OIDC_RP_CLIENT_ID | client id used for OIDC | impress |
| OIDC_RP_CLIENT_SECRET | client secret used for OIDC | |
| OIDC_OP_JWKS_ENDPOINT | JWKS endpoint for OIDC | |
| OIDC_OP_AUTHORIZATION_ENDPOINT | Autorization endpoint for OIDC | |
| OIDC_OP_TOKEN_ENDPOINT | Token endpoint for OIDC | |
| OIDC_OP_USER_ENDPOINT | User endpoint for OIDC | |
| OIDC_OP_LOGOUT_ENDPOINT | Logout endpoint for OIDC | |
| OIDC_AUTH_REQUEST_EXTRA_PARAMS | OIDC extra auth paramaters | {} |
| OIDC_RP_SCOPES | scopes requested for OIDC | openid email |
| LOGIN_REDIRECT_URL | login redirect url | |
| LOGIN_REDIRECT_URL_FAILURE | login redirect url on failure | |
| LOGOUT_REDIRECT_URL | logout redirect url | |
| OIDC_USE_NONCE | use nonce for OIDC | true |
| OIDC_REDIRECT_REQUIRE_HTTPS | Require https for OIDC redirect url | false |
| OIDC_REDIRECT_ALLOWED_HOSTS | Allowed hosts for OIDC redirect url | [] |
| OIDC_STORE_ID_TOKEN | Store OIDC token | true |
| OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION | faillback to email for identification | true |
| OIDC_ALLOW_DUPLICATE_EMAILS | Allow dupplicate emails | false |
| USER_OIDC_ESSENTIAL_CLAIMS | essential claims in OIDC token | [] |
| USER_OIDC_FIELDS_TO_FULLNAME | OIDC token claims to create full name | ["first_name", "last_name"] |
| USER_OIDC_FIELD_TO_SHORTNAME | OIDC token claims to create shortname | first_name |
| ALLOW_LOGOUT_GET_METHOD | Allow get logout method | true |
| AI_API_KEY | AI key to be used for AI Base url | |
| AI_BASE_URL | OpenAI compatible AI base url | |
| AI_MODEL | AI Model to use | |
| AI_ALLOW_REACH_FROM | Users that can use AI must be this level. options are "public", "authenticated", "restricted" | authenticated |
| Y_PROVIDER_API_KEY | Y provider API key | |
| Y_PROVIDER_API_BASE_URL | Y Provider url | |
| CONVERSION_API_ENDPOINT | Conversion API endpoint | convert-markdown |
| CONVERSION_API_CONTENT_FIELD | Conversion api content field | content |
| CONVERSION_API_TIMEOUT | Conversion api timeout | 30 |
| CONVERSION_API_SECURE | Require secure conversion api | false |
| LOGGING_LEVEL_LOGGERS_ROOT | default logging level. options are "DEBUG", "INFO", "WARN", "ERROR", "CRITICAL" | INFO |
| LOGGING_LEVEL_LOGGERS_APP | application logging level. options are "DEBUG", "INFO", "WARN", "ERROR", "CRITICAL" | INFO |
| API_USERS_LIST_LIMIT | Limit on API users | 5 |
| DJANGO_CSRF_TRUSTED_ORIGINS | CSRF trusted origins | [] |
| REDIS_URL | cache url | redis://redis:6379/1 |
| CACHES_DEFAULT_TIMEOUT | cache default timeout | 30 |

View File

@@ -1,21 +1,20 @@
# Installation on a k8s cluster
This document is a step-by-step guide that describes how to install Docs on a k8s cluster without AI features. It's a teaching document to learn how it's work. It needs to be adapt for production environment.
This document is a step-by-step guide that describes how to install Docs on a k8s cluster without AI features. It's a teaching document to learn how it works. It needs to be adapted for a production environment.
## Prerequisites
- k8s cluster with an nginx-ingress controller
- an OIDC provider (if you don't have one, we will provide an example)
- a PostgreSQL server (if you don't have one, we will provide an example)
- a Memcached server (if you don't have one, we will provide an example)
- a S3 bucket (if you don't have one, we will provide an example)
- an OIDC provider (if you don't have one, we provide an example)
- a PostgreSQL server (if you don't have one, we provide an example)
- a Memcached server (if you don't have one, we provide an example)
- a S3 bucket (if you don't have one, we provide an example)
### Test cluster
If you do not have a test cluster, you can install everything on a local kind cluster. In this case, the simplest way is to use our script **bin/start-kind.sh**.
If you do not have a test cluster, you can install everything on a local Kind cluster. In this case, the simplest way is to use our script **bin/start-kind.sh**.
To be able to use the script, you will need to install:
To be able to use the script, you need to install:
- Docker (https://docs.docker.com/desktop/)
- Kind (https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
@@ -23,7 +22,7 @@ To be able to use the script, you will need to install:
- Helm (https://helm.sh/docs/intro/quickstart/#install-helm)
```
./bin/start-kind.sh
./bin/start-kind.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 4700 100 4700 0 0 92867 0 --:--:-- --:--:-- --:--:-- 94000
@@ -46,11 +45,11 @@ It will expire on 24 March 2027 🗓
2. Create kind cluster with containerd registry config dir enabled
Creating cluster "suite" ...
✓ Ensuring node image (kindest/node:v1.27.3) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-suite"
You can now use your cluster with:
@@ -96,13 +95,14 @@ ingress-nginx-admission-create-t55ph 0/1 Completed 0 2m56s
ingress-nginx-admission-patch-94dvt 0/1 Completed 1 2m56s
ingress-nginx-controller-57c548c4cd-2rx47 1/1 Running 0 2m56s
```
When your k8s cluster is ready (the ingress nginx controller is up), you can start the deployment. This cluster is special because it uses the *.127.0.0.1.nip.io domain and mkcert certificates to have full HTTPS support and easy domain name management.
Please remember that *.127.0.0.1.nip.io will always resolve to 127.0.0.1, except in the k8s cluster where we configure CoreDNS to answer with the ingress-nginx service IP.
When your k8s cluster is ready (the ingress nginx controller is up), you can start the deployment. This cluster is special because it uses the `*.127.0.0.1.nip.io` domain and mkcert certificates to have full HTTPS support and easy domain name management.
Please remember that `*.127.0.0.1.nip.io` will always resolve to `127.0.0.1`, except in the k8s cluster where we configure CoreDNS to answer with the ingress-nginx service IP.
## Preparation
### What will you use to authenticate your users ?
### What do you use to authenticate your users?
Docs uses OIDC, so if you already have an OIDC provider, obtain the necessary information to use it. In the next step, we will see how to configure Django (and thus Docs) to use it. If you do not have a provider, we will show you how to deploy a local Keycloak instance (this is not a production deployment, just a demo).
@@ -117,9 +117,9 @@ keycloak-0 1/1 Running 0 6m48s
keycloak-postgresql-0 1/1 Running 0 6m48s
```
From here the important informations you will need are :
From here the important information you will need are:
```
```yaml
OIDC_OP_JWKS_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/certs
OIDC_OP_AUTHORIZATION_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/token
@@ -135,7 +135,7 @@ You can find these values in **examples/keycloak.values.yaml**
### Find redis server connexion values
Impress need a redis so we will start by deploying a redis :
Docs needs a redis so we start by deploying one:
```
$ helm install redis oci://registry-1.docker.io/bitnamicharts/redis -f examples/redis.values.yaml
@@ -148,7 +148,7 @@ redis-master-0 1/1 Running 0 35s
### Find postgresql connexion values
Impress uses a postgresql db as backend so if you have a provider, obtain the necessary information to use it. If you do not have, you can install a postgresql testing environment as follow:
Docs uses a postgresql database as backend, so if you have a provider, obtain the necessary information to use it. If you don't, you can install a postgresql testing environment as follow:
```
$ helm install postgresql oci://registry-1.docker.io/bitnamicharts/postgresql -f examples/postgresql.values.yaml
@@ -160,9 +160,9 @@ postgresql-0 1/1 Running 0 14m
redis-master-0 1/1 Running 0 42s
```
From here important informations you will need are :
From here the important information you will need are:
```
```yaml
DB_HOST: postgres-postgresql
DB_NAME: impress
DB_USER: dinum
@@ -175,7 +175,7 @@ POSTGRES_PASSWORD: pass
### Find s3 bucket connexion values
Impress uses a s3 bucket to store documents so if you have a provider obtain the necessary information to use it. If you do not have, you can install a local minio testing environment as follow:
Docs uses an s3 bucket to store documents, so if you have a provider obtain the necessary information to use it. If you don't, you can install a local minio testing environment as follow:
```
$ helm install minio oci://registry-1.docker.io/bitnamicharts/minio -f examples/minio.values.yaml
@@ -191,7 +191,7 @@ redis-master-0 1/1 Running 0 10m
## Deployment
Now you are ready to deploy Impress without AI. AI requiered more dependancies (openai API). To deploy impress you need to provide all previous informations to the helm chart.
Now you are ready to deploy Docs without AI. AI requires more dependencies (OpenAI API). To deploy Docs you need to provide all previous informations to the helm chart.
```
$ helm repo add impress https://suitenumerique.github.io/docs/
@@ -214,7 +214,7 @@ redis-master-0 1/1 Running 0 20m
## Test your deployment
In order to test your deployment you have to login to your instance. If you use exclusively our examples you can do :
In order to test your deployment you have to log into your instance. If you exclusively use our examples you can do:
```
$ kubectl get ingress
@@ -227,5 +227,4 @@ impress-docs-ws <none> impress.127.0.0.1.nip.io localhost
keycloak <none> keycloak.127.0.0.1.nip.io localhost 80 49m
```
You can use impress on https://impress.127.0.0.1.nip.io. The provisionning user in keycloak is impress/impress.
You can use Docs at https://impress.127.0.0.1.nip.io. The provisionning user in keycloak is impress/impress.

33
documentation/theming.md Normal file
View File

@@ -0,0 +1,33 @@
# Runtime Theming 🎨
### How to Use
To use this feature, simply set the `FRONTEND_CSS_URL` environment variable to the URL of your custom CSS file. For example:
```javascript
FRONTEND_CSS_URL=http://anything/custom-style.css
```
Once you've set this variable, our application will load your custom CSS file and apply the styles to our frontend application.
### Benefits
This feature provides several benefits, including:
* **Easy customization** 🔄: With this feature, you can easily customize the look and feel of our application without requiring any code changes.
* **Flexibility** 🌈: You can use any CSS styles you like to create a custom theme that meets your needs.
* **Runtime theming** ⏱️: This feature allows you to change the theme of our application at runtime, without requiring a restart or recompilation.
### Example Use Case
Let's say you want to change the background color of our application to a custom color. You can create a custom CSS file with the following contents:
```css
body {
background-color: #3498db;
}
```
Then, set the `FRONTEND_CSS_URL` environment variable to the URL of your custom CSS file. Once you've done this, our application will load your custom CSS file and apply the styles, changing the background color to the custom color you specified.

View File

@@ -50,15 +50,20 @@ OIDC_REDIRECT_ALLOWED_HOSTS=["http://localhost:8083", "http://localhost:3000"]
OIDC_AUTH_REQUEST_EXTRA_PARAMS={"acr_values": "eidas1"}
# AI
AI_FEATURE_ENABLED=true
AI_BASE_URL=https://openaiendpoint.com
AI_API_KEY=password
AI_MODEL=llama
# Collaboration
COLLABORATION_API_URL=http://nginx:8083/collaboration/api/
COLLABORATION_API_URL=http://y-provider:4444/collaboration/api/
COLLABORATION_BACKEND_BASE_URL=http://app-dev:8000
COLLABORATION_SERVER_ORIGIN=http://localhost:3000
COLLABORATION_SERVER_SECRET=my-secret
COLLABORATION_WS_URL=ws://localhost:8083/collaboration/ws/
COLLABORATION_WS_URL=ws://localhost:4444/collaboration/ws/
# Frontend
FRONTEND_THEME=dsfr
FRONTEND_THEME=default
FRONTEND_HOMEPAGE_FEATURE_ENABLED=True
FRONTEND_FOOTER_FEATURE_ENABLED=True
FRONTEND_URL_JSON_FOOTER=http://frontend:3000/contents/footer-demo.json

View File

@@ -151,6 +151,8 @@ class DocumentAdmin(TreeAdmin):
"path",
"depth",
"numchild",
"duplicated_from",
"attachments",
)
},
),
@@ -166,8 +168,10 @@ class DocumentAdmin(TreeAdmin):
"updated_at",
)
readonly_fields = (
"attachments",
"creator",
"depth",
"duplicated_from",
"id",
"numchild",
"path",

View File

@@ -1,6 +1,8 @@
"""Client serializers for the impress core app."""
import binascii
import mimetypes
from base64 import b64decode
from django.conf import settings
from django.db.models import Q
@@ -10,7 +12,7 @@ from django.utils.translation import gettext_lazy as _
import magic
from rest_framework import exceptions, serializers
from core import enums, models
from core import enums, models, utils
from core.services.ai_services import AI_ACTIONS
from core.services.converter_services import (
ConversionError,
@@ -27,6 +29,26 @@ class UserSerializer(serializers.ModelSerializer):
read_only_fields = ["id", "email", "full_name", "short_name"]
class UserLightSerializer(UserSerializer):
"""Serialize users with limited fields."""
id = serializers.SerializerMethodField(read_only=True)
email = serializers.SerializerMethodField(read_only=True)
def get_id(self, _user):
"""Return always None. Here to have the same fields than in UserSerializer."""
return None
def get_email(self, _user):
"""Return always None. Here to have the same fields than in UserSerializer."""
return None
class Meta:
model = models.User
fields = ["id", "email", "full_name", "short_name"]
read_only_fields = ["id", "email", "full_name", "short_name"]
class BaseAccessSerializer(serializers.ModelSerializer):
"""Serialize template accesses."""
@@ -118,6 +140,17 @@ class DocumentAccessSerializer(BaseAccessSerializer):
read_only_fields = ["id", "abilities"]
class DocumentAccessLightSerializer(DocumentAccessSerializer):
"""Serialize document accesses with limited fields."""
user = UserLightSerializer(read_only=True)
class Meta:
model = models.DocumentAccess
fields = ["id", "user", "team", "role", "abilities"]
read_only_fields = ["id", "team", "role", "abilities"]
class TemplateAccessSerializer(BaseAccessSerializer):
"""Serialize template accesses."""
@@ -268,6 +301,65 @@ class DocumentSerializer(ListDocumentSerializer):
return value
def validate_content(self, value):
"""Validate the content field."""
if not value:
return None
try:
b64decode(value, validate=True)
except binascii.Error as err:
raise serializers.ValidationError("Invalid base64 content.") from err
return value
def save(self, **kwargs):
"""
Process the content field to extract attachment keys and update the document's
"attachments" field for access control.
"""
content = self.validated_data.get("content", "")
extracted_attachments = set(utils.extract_attachments(content))
existing_attachments = (
set(self.instance.attachments or []) if self.instance else set()
)
new_attachments = extracted_attachments - existing_attachments
if new_attachments:
attachments_documents = (
models.Document.objects.filter(
attachments__overlap=list(new_attachments)
)
.only("path", "attachments")
.order_by("path")
)
user = self.context["request"].user
readable_per_se_paths = (
models.Document.objects.readable_per_se(user)
.order_by("path")
.values_list("path", flat=True)
)
readable_attachments_paths = utils.filter_descendants(
[doc.path for doc in attachments_documents],
readable_per_se_paths,
skip_sorting=True,
)
readable_attachments = set()
for document in attachments_documents:
if document.path not in readable_attachments_paths:
continue
readable_attachments.update(set(document.attachments) & new_attachments)
# Update attachments with readable keys
self.validated_data["attachments"] = list(
existing_attachments | readable_attachments
)
return super().save(**kwargs)
class ServerCreateDocumentSerializer(serializers.Serializer):
"""
@@ -381,6 +473,27 @@ class LinkDocumentSerializer(serializers.ModelSerializer):
]
class DocumentDuplicationSerializer(serializers.Serializer):
"""
Serializer for duplicating a document.
Allows specifying whether to keep access permissions.
"""
with_accesses = serializers.BooleanField(default=False)
def create(self, validated_data):
"""
This serializer is not intended to create objects.
"""
raise NotImplementedError("This serializer does not support creation.")
def update(self, instance, validated_data):
"""
This serializer is not intended to update objects.
"""
raise NotImplementedError("This serializer does not support updating.")
# Suppress the warning about not implementing `create` and `update` methods
# since we don't use a model and only rely on the serializer for validation
# pylint: disable=abstract-method

View File

@@ -2,7 +2,6 @@
# pylint: disable=too-many-lines
import logging
import re
import uuid
from urllib.parse import unquote, urlparse
@@ -17,6 +16,10 @@ from django.db import transaction
from django.db.models.expressions import RawSQL
from django.db.models.functions import Left, Length
from django.http import Http404, StreamingHttpResponse
from django.utils.decorators import method_decorator
from django.utils.text import capfirst
from django.utils.translation import gettext_lazy as _
from django.views.decorators.cache import cache_page
import requests
import rest_framework as drf
@@ -29,23 +32,14 @@ from rest_framework.throttling import UserRateThrottle
from core import authentication, enums, models
from core.services.ai_services import AIService
from core.services.collaboration_services import CollaborationService
from core.services.config_services import get_footer_json
from core.utils import extract_attachments, filter_descendants
from . import permissions, serializers, utils
from .filters import DocumentFilter, ListDocumentFilter
logger = logging.getLogger(__name__)
ATTACHMENTS_FOLDER = "attachments"
UUID_REGEX = (
r"[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}"
)
FILE_EXT_REGEX = r"\.[a-zA-Z0-9]{1,10}"
MEDIA_STORAGE_URL_PATTERN = re.compile(
f"{settings.MEDIA_URL:s}(?P<pk>{UUID_REGEX:s})/"
f"(?P<key>{ATTACHMENTS_FOLDER:s}/{UUID_REGEX:s}(?:-unsafe)?{FILE_EXT_REGEX:s})$"
)
COLLABORATION_WS_URL_PATTERN = re.compile(rf"(?:^|&)room=(?P<pk>{UUID_REGEX})(?:&|$)")
# pylint: disable=too-many-ancestors
@@ -389,10 +383,7 @@ class DocumentViewSet(
9. **Media Auth**: Authorize access to document media.
Example: GET /documents/media-auth/
10. **Collaboration Auth**: Authorize access to the collaboration server for a document.
Example: GET /documents/collaboration-auth/
11. **AI Transform**: Apply a transformation action on a piece of text with AI.
10. **AI Transform**: Apply a transformation action on a piece of text with AI.
Example: POST /documents/{id}/ai-transform/
Expected data:
- text (str): The input text.
@@ -400,7 +391,7 @@ class DocumentViewSet(
Returns: JSON response with the processed text.
Throttled by: AIDocumentRateThrottle, AIUserRateThrottle.
12. **AI Translate**: Translate a piece of text with AI.
11. **AI Translate**: Translate a piece of text with AI.
Example: POST /documents/{id}/ai-translate/
Expected data:
- text (str): The input text.
@@ -855,14 +846,15 @@ class DocumentViewSet(
)
# Get the highest readable ancestor
highest_readable = ancestors.readable_per_se(request.user).only("depth").first()
highest_readable = (
ancestors.readable_per_se(request.user).only("depth", "path").first()
)
if highest_readable is None:
raise (
drf.exceptions.PermissionDenied()
if request.user.is_authenticated
else drf.exceptions.NotAuthenticated()
)
paths_links_mapping = {}
ancestors_links = []
children_clause = db.Q()
@@ -885,6 +877,17 @@ class DocumentViewSet(
queryset = ancestors.filter(depth__gte=highest_readable.depth) | children
queryset = queryset.order_by("path")
# Annotate if the current document is the highest ancestor for the user
queryset = queryset.annotate(
is_highest_ancestor_for_user=db.Case(
db.When(
path=db.Value(highest_readable.path),
then=db.Value(True),
),
default=db.Value(False),
output_field=db.BooleanField(),
)
)
queryset = self.annotate_user_roles(queryset)
queryset = self.annotate_is_favorite(queryset)
@@ -902,6 +905,82 @@ class DocumentViewSet(
utils.nest_tree(serializer.data, self.queryset.model.steplen)
)
@drf.decorators.action(
detail=True,
methods=["post"],
permission_classes=[permissions.IsAuthenticated, permissions.AccessPermission],
url_path="duplicate",
)
@transaction.atomic
def duplicate(self, request, *args, **kwargs):
"""
Duplicate a document and store the links to attached files in the duplicated
document to allow cross-access.
Optionally duplicates accesses if `with_accesses` is set to true
in the payload.
"""
# Get document while checking permissions
document = self.get_object()
serializer = serializers.DocumentDuplicationSerializer(
data=request.data, partial=True
)
serializer.is_valid(raise_exception=True)
with_accesses = serializer.validated_data.get("with_accesses", False)
base64_yjs_content = document.content
# Duplicate the document instance
link_kwargs = (
{"link_reach": document.link_reach, "link_role": document.link_role}
if with_accesses
else {}
)
extracted_attachments = set(extract_attachments(document.content))
attachments = list(extracted_attachments & set(document.attachments))
duplicated_document = document.add_sibling(
"right",
title=capfirst(_("copy of {title}").format(title=document.title)),
content=base64_yjs_content,
attachments=attachments,
duplicated_from=document,
creator=request.user,
**link_kwargs,
)
# Always add the logged-in user as OWNER
accesses_to_create = [
models.DocumentAccess(
document=duplicated_document,
user=request.user,
role=models.RoleChoices.OWNER,
)
]
# If accesses should be duplicated, add other users' accesses as per original document
if with_accesses:
original_accesses = models.DocumentAccess.objects.filter(
document=document
).exclude(user=request.user)
accesses_to_create.extend(
models.DocumentAccess(
document=duplicated_document,
user_id=access.user_id,
team=access.team,
role=access.role,
)
for access in original_accesses
)
# Bulk create all the duplicated accesses
models.DocumentAccess.objects.bulk_create(accesses_to_create)
return drf_response.Response(
{"id": str(duplicated_document.id)}, status=status.HTTP_201_CREATED
)
@drf.decorators.action(detail=True, methods=["get"], url_path="versions")
def versions_list(self, request, *args, **kwargs):
"""
@@ -943,7 +1022,7 @@ class DocumentViewSet(
@drf.decorators.action(
detail=True,
methods=["get", "delete"],
url_path="versions/(?P<version_id>[0-9a-f-]{36})",
url_path="versions/(?P<version_id>[0-9a-z-]+)",
)
# pylint: disable=unused-argument
def versions_detail(self, request, pk, version_id, *args, **kwargs):
@@ -1051,7 +1130,7 @@ class DocumentViewSet(
# Generate a generic yet unique filename to store the image in object storage
file_id = uuid.uuid4()
extension = serializer.validated_data["expected_extension"]
ext = serializer.validated_data["expected_extension"]
# Prepare metadata for storage
extra_args = {
@@ -1063,7 +1142,7 @@ class DocumentViewSet(
extra_args["Metadata"]["is_unsafe"] = "true"
file_unsafe = "-unsafe"
key = f"{document.key_base}/{ATTACHMENTS_FOLDER:s}/{file_id!s}{file_unsafe}.{extension:s}"
key = f"{document.key_base}/{enums.ATTACHMENTS_FOLDER:s}/{file_id!s}{file_unsafe}.{ext:s}"
file_name = serializer.validated_data["file_name"]
if (
@@ -1083,15 +1162,19 @@ class DocumentViewSet(
file, default_storage.bucket_name, key, ExtraArgs=extra_args
)
# Make the attachment readable by document readers
document.attachments.append(key)
document.save()
return drf.response.Response(
{"file": f"{settings.MEDIA_URL:s}{key:s}"},
status=drf.status.HTTP_201_CREATED,
)
def _authorize_subrequest(self, request, pattern):
def _auth_get_original_url(self, request):
"""
Shared method to authorize access based on the original URL of an Nginx subrequest
and user permissions. Returns a dictionary of URL parameters if authorized.
Extracts and parses the original URL from the "HTTP_X_ORIGINAL_URL" header.
Raises PermissionDenied if the header is missing.
The original url is passed by nginx in the "HTTP_X_ORIGINAL_URL" header.
See corresponding ingress configuration in Helm chart and read about the
@@ -1102,14 +1185,6 @@ class DocumentViewSet(
to let this request go through (by returning a 200 code) or if we block it (by returning
a 403 error). Note that we return 403 errors without any further details for security
reasons.
Parameters:
- pattern: The regex pattern to extract identifiers from the URL.
Returns:
- A dictionary of URL parameters if the request is authorized.
Raises:
- PermissionDenied if authorization fails.
"""
# Extract the original URL from the request header
original_url = request.META.get("HTTP_X_ORIGINAL_URL")
@@ -1117,52 +1192,21 @@ class DocumentViewSet(
logger.debug("Missing HTTP_X_ORIGINAL_URL header in subrequest")
raise drf.exceptions.PermissionDenied()
parsed_url = urlparse(original_url)
match = pattern.search(parsed_url.path)
# If the path does not match the pattern, try to extract the parameters from the query
if not match:
match = pattern.search(parsed_url.query)
if not match:
logger.debug(
"Subrequest URL '%s' did not match pattern '%s'",
parsed_url.path,
pattern,
)
raise drf.exceptions.PermissionDenied()
logger.debug("Original url: '%s'", original_url)
return urlparse(original_url)
def _auth_get_url_params(self, pattern, fragment):
"""
Extracts URL parameters from the given fragment using the specified regex pattern.
Raises PermissionDenied if parameters cannot be extracted.
"""
match = pattern.search(fragment)
try:
url_params = match.groupdict()
return match.groupdict()
except (ValueError, AttributeError) as exc:
logger.debug("Failed to extract parameters from subrequest URL: %s", exc)
raise drf.exceptions.PermissionDenied() from exc
pk = url_params.get("pk")
if not pk:
logger.debug("Document ID (pk) not found in URL parameters: %s", url_params)
raise drf.exceptions.PermissionDenied()
# Fetch the document and check if the user has access
try:
document = models.Document.objects.get(pk=pk)
except models.Document.DoesNotExist as exc:
logger.debug("Document with ID '%s' does not exist", pk)
raise drf.exceptions.PermissionDenied() from exc
user_abilities = document.get_abilities(request.user)
if not user_abilities.get(self.action, False):
logger.debug(
"User '%s' lacks permission for document '%s'", request.user, pk
)
raise drf.exceptions.PermissionDenied()
logger.debug(
"Subrequest authorization successful. Extracted parameters: %s", url_params
)
return url_params, user_abilities, request.user.id
@drf.decorators.action(detail=False, methods=["get"], url_path="media-auth")
def media_auth(self, request, *args, **kwargs):
"""
@@ -1174,36 +1218,42 @@ class DocumentViewSet(
annotation. The request will then be proxied to the object storage backend who will
respond with the file after checking the signature included in headers.
"""
url_params, _, _ = self._authorize_subrequest(
request, MEDIA_STORAGE_URL_PATTERN
parsed_url = self._auth_get_original_url(request)
url_params = self._auth_get_url_params(
enums.MEDIA_STORAGE_URL_PATTERN, parsed_url.path
)
pk, key = url_params.values()
user = request.user
key = f"{url_params['pk']:s}/{url_params['attachment']:s}"
# Look for a document to which the user has access and that includes this attachment
# We must look into all descendants of any document to which the user has access per se
readable_per_se_paths = (
self.queryset.readable_per_se(user)
.order_by("path")
.values_list("path", flat=True)
)
attachments_documents = (
self.queryset.filter(attachments__contains=[key])
.only("path")
.order_by("path")
)
readable_attachments_paths = filter_descendants(
[doc.path for doc in attachments_documents],
readable_per_se_paths,
skip_sorting=True,
)
if not readable_attachments_paths:
logger.debug("User '%s' lacks permission for attachment", user)
raise drf.exceptions.PermissionDenied()
# Generate S3 authorization headers using the extracted URL parameters
request = utils.generate_s3_authorization_headers(f"{pk:s}/{key:s}")
request = utils.generate_s3_authorization_headers(key)
return drf.response.Response("authorized", headers=request.headers, status=200)
@drf.decorators.action(detail=False, methods=["get"], url_path="collaboration-auth")
def collaboration_auth(self, request, *args, **kwargs):
"""
This view is used by an Nginx subrequest to control access to a document's
collaboration server.
"""
_, user_abilities, user_id = self._authorize_subrequest(
request, COLLABORATION_WS_URL_PATTERN
)
can_edit = user_abilities["partial_update"]
# Add the collaboration server secret token to the headers
headers = {
"Authorization": settings.COLLABORATION_SERVER_SECRET,
"X-Can-Edit": str(can_edit),
"X-User-Id": str(user_id),
}
return drf.response.Response("authorized", headers=headers, status=200)
@drf.decorators.action(
detail=True,
methods=["post"],
@@ -1323,12 +1373,7 @@ class DocumentViewSet(
class DocumentAccessViewSet(
ResourceAccessViewsetMixin,
drf.mixins.CreateModelMixin,
drf.mixins.DestroyModelMixin,
drf.mixins.ListModelMixin,
drf.mixins.RetrieveModelMixin,
drf.mixins.UpdateModelMixin,
viewsets.GenericViewSet,
viewsets.ModelViewSet,
):
"""
API ViewSet for all interactions with document accesses.
@@ -1360,6 +1405,32 @@ class DocumentAccessViewSet(
queryset = models.DocumentAccess.objects.select_related("user").all()
resource_field_name = "document"
serializer_class = serializers.DocumentAccessSerializer
is_current_user_owner_or_admin = False
def get_queryset(self):
"""Return the queryset according to the action."""
queryset = super().get_queryset()
if self.action == "list":
try:
document = models.Document.objects.get(pk=self.kwargs["resource_id"])
except models.Document.DoesNotExist:
return queryset.none()
roles = set(document.get_roles(self.request.user))
is_owner_or_admin = bool(roles.intersection(set(models.PRIVILEGED_ROLES)))
self.is_current_user_owner_or_admin = is_owner_or_admin
if not is_owner_or_admin:
# Return only the document owner access
queryset = queryset.filter(role__in=models.PRIVILEGED_ROLES)
return queryset
def get_serializer_class(self):
if self.action == "list" and not self.is_current_user_owner_or_admin:
return serializers.DocumentAccessLightSerializer
return super().get_serializer_class()
def perform_create(self, serializer):
"""Add a new access to the document and send an email to the new added user."""
@@ -1616,9 +1687,13 @@ class ConfigView(drf.views.APIView):
Return a dictionary of public settings.
"""
array_settings = [
"AI_FEATURE_ENABLED",
"COLLABORATION_WS_URL",
"CRISP_WEBSITE_ID",
"ENVIRONMENT",
"FRONTEND_CSS_URL",
"FRONTEND_HOMEPAGE_FEATURE_ENABLED",
"FRONTEND_FOOTER_FEATURE_ENABLED",
"FRONTEND_THEME",
"MEDIA_BASE_URL",
"POSTHOG_KEY",
@@ -1632,3 +1707,22 @@ class ConfigView(drf.views.APIView):
dict_settings[setting] = getattr(settings, setting)
return drf.response.Response(dict_settings)
class FooterView(drf.views.APIView):
"""API ViewSet for sharing the footer JSON."""
permission_classes = [AllowAny]
@method_decorator(cache_page(settings.FRONTEND_FOOTER_VIEW_CACHE_TIMEOUT))
def get(self, request):
"""
GET /api/v1.0/footer/
Return the footer JSON.
"""
json_footer = (
get_footer_json(settings.FRONTEND_URL_JSON_FOOTER)
if settings.FRONTEND_URL_JSON_FOOTER
else {}
)
return drf.response.Response(json_footer)

View File

@@ -2,10 +2,26 @@
Core application enums declaration
"""
from django.conf import global_settings
import re
from django.conf import global_settings, settings
from django.db import models
from django.utils.translation import gettext_lazy as _
ATTACHMENTS_FOLDER = "attachments"
UUID_REGEX = (
r"[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}"
)
FILE_EXT_REGEX = r"\.[a-zA-Z0-9]{1,10}"
MEDIA_STORAGE_URL_PATTERN = re.compile(
f"{settings.MEDIA_URL:s}(?P<pk>{UUID_REGEX:s})/"
f"(?P<attachment>{ATTACHMENTS_FOLDER:s}/{UUID_REGEX:s}(?:-unsafe)?{FILE_EXT_REGEX:s})$"
)
MEDIA_STORAGE_URL_EXTRACT = re.compile(
f"{settings.MEDIA_URL:s}({UUID_REGEX}/{ATTACHMENTS_FOLDER}/{UUID_REGEX}{FILE_EXT_REGEX})"
)
# In Django's code base, `LANGUAGES` is set by default with all supported languages.
# We can use it for the choice of languages which should not be limited to the few languages
# active in the app.

View File

@@ -13,6 +13,22 @@ from core import models
fake = Faker()
YDOC_HELLO_WORLD_BASE64 = (
"AR717vLVDgAHAQ5kb2N1bWVudC1zdG9yZQMKYmxvY2tHcm91cAcA9e7y1Q4AAw5ibG9ja0NvbnRh"
"aW5lcgcA9e7y1Q4BAwdoZWFkaW5nBwD17vLVDgIGBgD17vLVDgMGaXRhbGljAnt9hPXu8tUOBAVI"
"ZWxsb4b17vLVDgkGaXRhbGljBG51bGwoAPXu8tUOAg10ZXh0QWxpZ25tZW50AXcEbGVmdCgA9e7y"
"1Q4CBWxldmVsAX0BKAD17vLVDgECaWQBdyQwNGQ2MjM0MS04MzI2LTQyMzYtYTA4My00ODdlMjZm"
"YWQyMzAoAPXu8tUOAQl0ZXh0Q29sb3IBdwdkZWZhdWx0KAD17vLVDgEPYmFja2dyb3VuZENvbG9y"
"AXcHZGVmYXVsdIf17vLVDgEDDmJsb2NrQ29udGFpbmVyBwD17vLVDhADDmJ1bGxldExpc3RJdGVt"
"BwD17vLVDhEGBAD17vLVDhIBd4b17vLVDhMEYm9sZAJ7fYT17vLVDhQCb3KG9e7y1Q4WBGJvbGQE"
"bnVsbIT17vLVDhcCbGQoAPXu8tUOEQ10ZXh0QWxpZ25tZW50AXcEbGVmdCgA9e7y1Q4QAmlkAXck"
"ZDM1MWUwNjgtM2U1NS00MjI2LThlYTUtYWJiMjYzMTk4ZTJhKAD17vLVDhAJdGV4dENvbG9yAXcH"
"ZGVmYXVsdCgA9e7y1Q4QD2JhY2tncm91bmRDb2xvcgF3B2RlZmF1bHSH9e7y1Q4QAw5ibG9ja0Nv"
"bnRhaW5lcgcA9e7y1Q4eAwlwYXJhZ3JhcGgoAPXu8tUOHw10ZXh0QWxpZ25tZW50AXcEbGVmdCgA"
"9e7y1Q4eAmlkAXckODk3MDBjMDctZTBlMS00ZmUwLWFjYTItODQ5MzIwOWE3ZTQyKAD17vLVDh4J"
"dGV4dENvbG9yAXcHZGVmYXVsdCgA9e7y1Q4eD2JhY2tncm91bmRDb2xvcgF3B2RlZmF1bHQA"
)
class UserFactory(factory.django.DjangoModelFactory):
"""A factory to random users for testing purposes."""
@@ -75,7 +91,7 @@ class DocumentFactory(factory.django.DjangoModelFactory):
title = factory.Sequence(lambda n: f"document{n}")
excerpt = factory.Sequence(lambda n: f"excerpt{n}")
content = factory.Sequence(lambda n: f"content{n}")
content = YDOC_HELLO_WORLD_BASE64
creator = factory.SubFactory(UserFactory)
deleted_at = None
link_reach = factory.fuzzy.FuzzyChoice(

View File

@@ -0,0 +1,77 @@
# Generated by Django 5.1.4 on 2025-01-18 11:53
import re
import django.contrib.postgres.fields
import django.db.models.deletion
from django.core.files.storage import default_storage
from django.db import migrations, models
from botocore.exceptions import ClientError
import core.models
from core.utils import extract_attachments
def populate_attachments_on_all_documents(apps, schema_editor):
"""Populate "attachments" field on all existing documents in the database."""
Document = apps.get_model("core", "Document")
for document in Document.objects.all():
try:
response = default_storage.connection.meta.client.get_object(
Bucket=default_storage.bucket_name, Key=f"{document.pk!s}/file"
)
except (FileNotFoundError, ClientError):
pass
else:
content = response["Body"].read().decode("utf-8")
document.attachments = extract_attachments(content)
document.save(update_fields=["attachments"])
class Migration(migrations.Migration):
dependencies = [
("core", "0019_alter_user_language_default_to_null"),
]
operations = [
# v2.0.0 was released so we can now remove BC field "is_public"
migrations.RemoveField(
model_name="document",
name="is_public",
),
migrations.AlterModelManagers(
name="user",
managers=[
("objects", core.models.UserManager()),
],
),
migrations.AddField(
model_name="document",
name="attachments",
field=django.contrib.postgres.fields.ArrayField(
base_field=models.CharField(max_length=255),
blank=True,
default=list,
editable=False,
null=True,
size=None,
),
),
migrations.AddField(
model_name="document",
name="duplicated_from",
field=models.ForeignKey(
blank=True,
editable=False,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="duplicates",
to="core.document",
),
),
migrations.RunPython(
populate_attachments_on_all_documents,
reverse_code=migrations.RunPython.noop,
),
]

View File

@@ -13,6 +13,7 @@ from logging import getLogger
from django.conf import settings
from django.contrib.auth import models as auth_models
from django.contrib.auth.base_user import AbstractBaseUser
from django.contrib.postgres.fields import ArrayField
from django.contrib.sites.models import Site
from django.core import mail, validators
from django.core.cache import cache
@@ -23,7 +24,7 @@ from django.db import models, transaction
from django.db.models.functions import Left, Length
from django.template.loader import render_to_string
from django.utils import timezone
from django.utils.functional import cached_property, lazy
from django.utils.functional import cached_property
from django.utils.translation import get_language, override
from django.utils.translation import gettext_lazy as _
@@ -96,7 +97,7 @@ class LinkReachChoices(models.TextChoices):
"""
# If no ancestors, return all options
if not ancestors_links:
return {reach: LinkRoleChoices.values for reach in cls.values}
return dict.fromkeys(cls.values, LinkRoleChoices.values)
# Initialize result with all possible reaches and role options as sets
result = {reach: set(LinkRoleChoices.values) for reach in cls.values}
@@ -243,7 +244,7 @@ class User(AbstractBaseUser, BaseModel, auth_models.PermissionsMixin):
language = models.CharField(
max_length=10,
choices=lazy(lambda: settings.LANGUAGES, tuple)(),
choices=settings.LANGUAGES,
default=None,
verbose_name=_("language"),
help_text=_("The language in which the user wants to see the interface."),
@@ -363,10 +364,9 @@ class BaseAccess(BaseModel):
class Meta:
abstract = True
def _get_abilities(self, resource, user):
def _get_roles(self, resource, user):
"""
Compute and return abilities for a given user taking into account
the current state of the object.
Get the roles a user has on a resource.
"""
roles = []
if user.is_authenticated:
@@ -381,6 +381,15 @@ class BaseAccess(BaseModel):
except (self._meta.model.DoesNotExist, IndexError):
roles = []
return roles
def _get_abilities(self, resource, user):
"""
Compute and return abilities for a given user taking into account
the current state of the object.
"""
roles = self._get_roles(resource, user)
is_owner_or_admin = bool(
set(roles).intersection({RoleChoices.OWNER, RoleChoices.ADMIN})
)
@@ -427,10 +436,12 @@ class DocumentQuerySet(MP_NodeQuerySet):
def readable_per_se(self, user):
"""
Filters the queryset to return documents that the given user has
permission to read.
Filters the queryset to return documents on which the given user has
direct access, team access or link access. This will not return all the
documents that a user can read because it can be obtained via an ancestor.
:param user: The user for whom readable documents are to be fetched.
:return: A queryset of documents readable by the user.
:return: A queryset of documents for which the user has direct access,
team access or link access.
"""
if user.is_authenticated:
return self.filter(
@@ -442,26 +453,15 @@ class DocumentQuerySet(MP_NodeQuerySet):
return self.filter(link_reach=LinkReachChoices.PUBLIC)
class DocumentManager(MP_NodeManager):
class DocumentManager(MP_NodeManager.from_queryset(DocumentQuerySet)):
"""
Custom manager for the Document model, enabling the use of the custom
queryset methods directly from the model manager.
"""
def get_queryset(self):
"""
Overrides the default get_queryset method to return a custom queryset.
:return: An instance of DocumentQuerySet.
"""
return DocumentQuerySet(self.model, using=self._db)
def readable_per_se(self, user):
"""
Filters documents based on user permissions using the custom queryset.
:param user: The user for whom readable documents are to be fetched.
:return: A queryset of documents readable by the user.
"""
return self.get_queryset().readable_per_se(user)
"""Sets the custom queryset as the default."""
return self._queryset_class(self.model).order_by("path")
class Document(MP_Node, BaseModel):
@@ -486,6 +486,21 @@ class Document(MP_Node, BaseModel):
)
deleted_at = models.DateTimeField(null=True, blank=True)
ancestors_deleted_at = models.DateTimeField(null=True, blank=True)
duplicated_from = models.ForeignKey(
"self",
on_delete=models.SET_NULL,
related_name="duplicates",
editable=False,
blank=True,
null=True,
)
attachments = ArrayField(
models.CharField(max_length=255),
default=list,
editable=False,
blank=True,
null=True,
)
_content = None
@@ -582,9 +597,13 @@ class Document(MP_Node, BaseModel):
def get_content_response(self, version_id=""):
"""Get the content in a specific version of the document"""
return default_storage.connection.meta.client.get_object(
Bucket=default_storage.bucket_name, Key=self.file_key, VersionId=version_id
)
params = {
"Bucket": default_storage.bucket_name,
"Key": self.file_key,
}
if version_id:
params["VersionId"] = version_id
return default_storage.connection.meta.client.get_object(**params)
def get_versions_slice(self, from_version_id="", min_datetime=None, page_size=None):
"""Get document versions from object storage with pagination and starting conditions"""
@@ -730,6 +749,32 @@ class Document(MP_Node, BaseModel):
return dict(links_definitions) # Convert defaultdict back to a normal dict
def compute_ancestors_links(self, user):
"""
Compute the ancestors links for the current document up to the highest readable ancestor.
"""
ancestors = (
(self.get_ancestors() | self._meta.model.objects.filter(pk=self.pk))
.filter(ancestors_deleted_at__isnull=True)
.order_by("path")
)
highest_readable = ancestors.readable_per_se(user).only("depth").first()
if highest_readable is None:
return []
ancestors_links = []
paths_links_mapping = {}
for ancestor in ancestors.filter(depth__gte=highest_readable.depth):
ancestors_links.append(
{"link_reach": ancestor.link_reach, "link_role": ancestor.link_role}
)
paths_links_mapping[ancestor.path] = ancestors_links.copy()
ancestors_links = paths_links_mapping.get(self.path[: -self.steplen], [])
return ancestors_links
def get_abilities(self, user, ancestors_links=None):
"""
Compute and return abilities for a given user on the document.
@@ -737,7 +782,7 @@ class Document(MP_Node, BaseModel):
if self.depth <= 1 or getattr(self, "is_highest_ancestor_for_user", False):
ancestors_links = []
elif ancestors_links is None:
ancestors_links = self.get_ancestors().values("link_reach", "link_role")
ancestors_links = self.compute_ancestors_links(user=user)
roles = set(
self.get_roles(user)
@@ -796,6 +841,7 @@ class Document(MP_Node, BaseModel):
"cors_proxy": can_get,
"descendants": can_get,
"destroy": is_owner,
"duplicate": can_get,
"favorite": can_get and user.is_authenticated,
"link_configuration": is_owner_or_admin,
"invite_owner": is_owner,
@@ -1065,7 +1111,41 @@ class DocumentAccess(BaseAccess):
"""
Compute and return abilities for a given user on the document access.
"""
return self._get_abilities(self.document, user)
roles = self._get_roles(self.document, user)
is_owner_or_admin = bool(set(roles).intersection(set(PRIVILEGED_ROLES)))
if self.role == RoleChoices.OWNER:
can_delete = (
RoleChoices.OWNER in roles
and self.document.accesses.filter(role=RoleChoices.OWNER).count() > 1
)
set_role_to = (
[RoleChoices.ADMIN, RoleChoices.EDITOR, RoleChoices.READER]
if can_delete
else []
)
else:
can_delete = is_owner_or_admin
set_role_to = []
if RoleChoices.OWNER in roles:
set_role_to.append(RoleChoices.OWNER)
if is_owner_or_admin:
set_role_to.extend(
[RoleChoices.ADMIN, RoleChoices.EDITOR, RoleChoices.READER]
)
# Remove the current role as we don't want to propose it as an option
try:
set_role_to.remove(self.role)
except ValueError:
pass
return {
"destroy": can_delete,
"update": bool(set_role_to) and is_owner_or_admin,
"partial_update": bool(set_role_to) and is_owner_or_admin,
"retrieve": self.user and self.user.id == user.id or is_owner_or_admin,
"set_role_to": set_role_to,
}
class Template(BaseModel):

View File

@@ -0,0 +1,25 @@
"""Config services."""
import logging
import requests
logger = logging.getLogger(__name__)
def get_footer_json(footer_json_url: str) -> dict:
"""
Fetches the footer JSON from the given URL."
"""
try:
response = requests.get(
footer_json_url, timeout=5, headers={"User-Agent": "Docs-Application"}
)
response.raise_for_status()
footer_json = response.json()
return footer_json
except (requests.RequestException, ValueError) as e:
logger.error("Failed to fetch footer JSON: %s", e)
return {}

View File

@@ -2,6 +2,8 @@
from unittest import mock
from django.core.cache import cache
import pytest
USER = "user"
@@ -9,6 +11,12 @@ TEAM = "team"
VIA = [USER, TEAM]
@pytest.fixture(autouse=True)
def clear_cache():
"""Fixture to clear the cache before each test."""
cache.clear()
@pytest.fixture
def mock_user_teams():
"""Mock for the "teams" property on the User model."""

View File

@@ -59,8 +59,32 @@ def test_api_document_accesses_list_authenticated_unrelated():
}
def test_api_document_accesses_list_unexisting_document():
"""
Listing document accesses for an unexisting document should return an empty list.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
response = client.get(f"/api/v1.0/documents/{uuid4()!s}/accesses/")
assert response.status_code == 200
assert response.json() == {
"count": 0,
"next": None,
"previous": None,
"results": [],
}
@pytest.mark.parametrize("via", VIA)
def test_api_document_accesses_list_authenticated_related(via, mock_user_teams):
@pytest.mark.parametrize(
"role", [role for role in models.RoleChoices if role not in models.PRIVILEGED_ROLES]
)
def test_api_document_accesses_list_authenticated_related_non_privileged(
via, role, mock_user_teams
):
"""
Authenticated users should be able to list document accesses for a document
to which they are directly related, whatever their role in the document.
@@ -70,24 +94,114 @@ def test_api_document_accesses_list_authenticated_related(via, mock_user_teams):
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory()
owner = factories.UserFactory()
accesses = []
document_access = factories.UserDocumentAccessFactory(
user=owner, role=models.RoleChoices.OWNER
)
accesses.append(document_access)
document = document_access.document
if via == USER:
models.DocumentAccess.objects.create(
document=document,
user=user,
role=role,
)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
models.DocumentAccess.objects.create(
document=document,
team="lasuite",
role=role,
)
access1 = factories.TeamDocumentAccessFactory(document=document)
access2 = factories.UserDocumentAccessFactory(document=document)
accesses.append(access1)
accesses.append(access2)
# Accesses for other documents to which the user is related should not be listed either
other_access = factories.UserDocumentAccessFactory(user=user)
factories.UserDocumentAccessFactory(document=other_access.document)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/accesses/",
)
# Return only owners
owners_accesses = [
access for access in accesses if access.role in models.PRIVILEGED_ROLES
]
assert response.status_code == 200
content = response.json()
assert content["count"] == len(owners_accesses)
assert sorted(content["results"], key=lambda x: x["id"]) == sorted(
[
{
"id": str(access.id),
"user": {
"id": None,
"email": None,
"full_name": access.user.full_name,
"short_name": access.user.short_name,
}
if access.user
else None,
"team": access.team,
"role": access.role,
"abilities": access.get_abilities(user),
}
for access in owners_accesses
],
key=lambda x: x["id"],
)
for access in content["results"]:
assert access["role"] in models.PRIVILEGED_ROLES
@pytest.mark.parametrize("via", VIA)
@pytest.mark.parametrize("role", models.PRIVILEGED_ROLES)
def test_api_document_accesses_list_authenticated_related_privileged_roles(
via, role, mock_user_teams
):
"""
Authenticated users should be able to list document accesses for a document
to which they are directly related, whatever their role in the document.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
owner = factories.UserFactory()
accesses = []
document_access = factories.UserDocumentAccessFactory(
user=owner, role=models.RoleChoices.OWNER
)
accesses.append(document_access)
document = document_access.document
user_access = None
if via == USER:
user_access = models.DocumentAccess.objects.create(
document=document,
user=user,
role=random.choice(models.RoleChoices.values),
role=role,
)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
user_access = models.DocumentAccess.objects.create(
document=document,
team="lasuite",
role=random.choice(models.RoleChoices.values),
role=role,
)
access1 = factories.TeamDocumentAccessFactory(document=document)
access2 = factories.UserDocumentAccessFactory(document=document)
accesses.append(access1)
accesses.append(access2)
# Accesses for other documents to which the user is related should not be listed either
other_access = factories.UserDocumentAccessFactory(user=user)
@@ -102,7 +216,7 @@ def test_api_document_accesses_list_authenticated_related(via, mock_user_teams):
assert response.status_code == 200
content = response.json()
assert len(content["results"]) == 3
assert len(content["results"]) == 4
assert sorted(content["results"], key=lambda x: x["id"]) == sorted(
[
{
@@ -126,6 +240,13 @@ def test_api_document_accesses_list_authenticated_related(via, mock_user_teams):
"role": access2.role,
"abilities": access2.get_abilities(user),
},
{
"id": str(document_access.id),
"user": serializers.UserSerializer(instance=owner).data,
"team": "",
"role": models.RoleChoices.OWNER,
"abilities": document_access.get_abilities(user),
},
],
key=lambda x: x["id"],
)
@@ -184,7 +305,10 @@ def test_api_document_accesses_retrieve_authenticated_unrelated():
@pytest.mark.parametrize("via", VIA)
def test_api_document_accesses_retrieve_authenticated_related(via, mock_user_teams):
@pytest.mark.parametrize("role", models.RoleChoices)
def test_api_document_accesses_retrieve_authenticated_related(
via, role, mock_user_teams
):
"""
A user who is related to a document should be allowed to retrieve the
associated document user accesses.
@@ -196,10 +320,12 @@ def test_api_document_accesses_retrieve_authenticated_related(via, mock_user_tea
document = factories.DocumentFactory()
if via == USER:
factories.UserDocumentAccessFactory(document=document, user=user)
factories.UserDocumentAccessFactory(document=document, user=user, role=role)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamDocumentAccessFactory(document=document, team="lasuite")
factories.TeamDocumentAccessFactory(
document=document, team="lasuite", role=role
)
access = factories.UserDocumentAccessFactory(document=document)
@@ -207,16 +333,19 @@ def test_api_document_accesses_retrieve_authenticated_related(via, mock_user_tea
f"/api/v1.0/documents/{document.id!s}/accesses/{access.id!s}/",
)
access_user = serializers.UserSerializer(instance=access.user).data
if not role in models.PRIVILEGED_ROLES:
assert response.status_code == 403
else:
access_user = serializers.UserSerializer(instance=access.user).data
assert response.status_code == 200
assert response.json() == {
"id": str(access.id),
"user": access_user,
"team": "",
"role": access.role,
"abilities": access.get_abilities(user),
}
assert response.status_code == 200
assert response.json() == {
"id": str(access.id),
"user": access_user,
"team": "",
"role": access.role,
"abilities": access.get_abilities(user),
}
def test_api_document_accesses_update_anonymous():

View File

@@ -5,7 +5,6 @@ Test AI transform API endpoint for users in impress's core app.
import random
from unittest.mock import MagicMock, patch
from django.core.cache import cache
from django.test import override_settings
import pytest
@@ -17,12 +16,6 @@ from core.tests.conftest import TEAM, USER, VIA
pytestmark = pytest.mark.django_db
@pytest.fixture(autouse=True)
def clear_cache():
"""Fixture to clear the cache before each test."""
cache.clear()
@pytest.fixture
def ai_settings():
"""Fixture to set AI settings."""

View File

@@ -5,7 +5,6 @@ Test AI translate API endpoint for users in impress's core app.
import random
from unittest.mock import MagicMock, patch
from django.core.cache import cache
from django.test import override_settings
import pytest
@@ -17,12 +16,6 @@ from core.tests.conftest import TEAM, USER, VIA
pytestmark = pytest.mark.django_db
@pytest.fixture(autouse=True)
def clear_cache():
"""Fixture to clear the cache before each test."""
cache.clear()
@pytest.fixture
def ai_settings():
"""Fixture to set AI settings."""

View File

@@ -67,10 +67,12 @@ def test_api_documents_attachment_upload_anonymous_success():
file_path = response.json()["file"]
match = pattern.search(file_path)
file_id = match.group(1)
# Validate that file_id is a valid UUID
uuid.UUID(file_id)
document.refresh_from_db()
assert document.attachments == [f"{document.id!s}/attachments/{file_id!s}.png"]
# Now, check the metadata of the uploaded file
key = file_path.replace("/media", "")
file_head = default_storage.connection.meta.client.head_object(
@@ -112,6 +114,9 @@ def test_api_documents_attachment_upload_authenticated_forbidden(reach, role):
"detail": "You do not have permission to perform this action."
}
document.refresh_from_db()
assert document.attachments == []
@pytest.mark.parametrize(
"reach, role",
@@ -122,8 +127,8 @@ def test_api_documents_attachment_upload_authenticated_forbidden(reach, role):
)
def test_api_documents_attachment_upload_authenticated_success(reach, role):
"""
Autenticated who are not related to a document should be able to upload a file
if the link reach and role permit it.
Autenticated users who are not related to a document should be able to upload
a file when the link reach and role permit it.
"""
user = factories.UserFactory()
@@ -145,6 +150,9 @@ def test_api_documents_attachment_upload_authenticated_success(reach, role):
# Validate that file_id is a valid UUID
uuid.UUID(file_id)
document.refresh_from_db()
assert document.attachments == [f"{document.id!s}/attachments/{file_id!s}.png"]
@pytest.mark.parametrize("via", VIA)
def test_api_documents_attachment_upload_reader(via, mock_user_teams):
@@ -175,6 +183,9 @@ def test_api_documents_attachment_upload_reader(via, mock_user_teams):
"detail": "You do not have permission to perform this action."
}
document.refresh_from_db()
assert document.attachments == []
@pytest.mark.parametrize("role", ["editor", "administrator", "owner"])
@pytest.mark.parametrize("via", VIA)
@@ -211,6 +222,9 @@ def test_api_documents_attachment_upload_success(via, role, mock_user_teams):
# Validate that file_id is a valid UUID
uuid.UUID(file_id)
document.refresh_from_db()
assert document.attachments == [f"{document.id!s}/attachments/{file_id!s}.png"]
# Now, check the metadata of the uploaded file
key = file_path.replace("/media", "")
file_head = default_storage.connection.meta.client.head_object(
@@ -236,6 +250,9 @@ def test_api_documents_attachment_upload_invalid(client):
assert response.status_code == 400
assert response.json() == {"file": ["No file was submitted."]}
document.refresh_from_db()
assert document.attachments == []
def test_api_documents_attachment_upload_size_limit_exceeded(settings):
"""The uploaded file should not exceeed the maximum size in settings."""
@@ -258,6 +275,9 @@ def test_api_documents_attachment_upload_size_limit_exceeded(settings):
assert response.status_code == 400
assert response.json() == {"file": ["File size exceeds the maximum limit of 1 MB."]}
document.refresh_from_db()
assert document.attachments == []
@pytest.mark.parametrize(
"name,content,extension,content_type",
@@ -293,6 +313,11 @@ def test_api_documents_attachment_upload_fix_extension(
match = pattern.search(file_path)
file_id = match.group(1)
document.refresh_from_db()
assert document.attachments == [
f"{document.id!s}/attachments/{file_id!s}.{extension:s}"
]
assert "-unsafe" in file_id
# Validate that file_id is a valid UUID
file_id = file_id.replace("-unsafe", "")
@@ -323,6 +348,9 @@ def test_api_documents_attachment_upload_empty_file():
assert response.status_code == 400
assert response.json() == {"file": ["The submitted file is empty."]}
document.refresh_from_db()
assert document.attachments == []
def test_api_documents_attachment_upload_unsafe():
"""A file with an unsafe mime type should be tagged as such."""
@@ -345,6 +373,9 @@ def test_api_documents_attachment_upload_unsafe():
match = pattern.search(file_path)
file_id = match.group(1)
document.refresh_from_db()
assert document.attachments == [f"{document.id!s}/attachments/{file_id!s}.exe"]
assert "-unsafe" in file_id
# Validate that file_id is a valid UUID
file_id = file_id.replace("-unsafe", "")

View File

@@ -0,0 +1,207 @@
"""
Test file uploads API endpoint for users in impress's core app.
"""
import base64
import uuid
from io import BytesIO
from urllib.parse import urlparse
from django.conf import settings
from django.core.files.storage import default_storage
from django.utils import timezone
import pycrdt
import pytest
import requests
from rest_framework.test import APIClient
from core import factories, models
pytestmark = pytest.mark.django_db
PIXEL = (
b"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x01\x00\x00\x00\x01\x08\x06\x00"
b"\x00\x00\x1f\x15\xc4\x89\x00\x00\x00\nIDATx\x9cc\xf8\xff\xff?\x00\x05\xfe\x02\xfe"
b"\xa7V\xbd\xfa\x00\x00\x00\x00IEND\xaeB`\x82"
)
def get_image_refs(document_id):
"""Generate an image key for testing."""
image_key = f"{document_id!s}/attachments/{uuid.uuid4()!s}.png"
default_storage.connection.meta.client.put_object(
Bucket=default_storage.bucket_name,
Key=image_key,
Body=BytesIO(PIXEL),
ContentType="image/png",
)
return image_key, f"http://localhost/media/{image_key:s}"
def test_api_documents_duplicate_forbidden():
"""A user who doesn't have read access to a document should not be allowed to duplicate it."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(
link_reach="restricted",
users=[factories.UserFactory()],
title="my document",
)
response = client.post(f"/api/v1.0/documents/{document.id!s}/duplicate/")
assert response.status_code == 403
assert models.Document.objects.count() == 1
def test_api_documents_duplicate_anonymous():
"""Anonymous users should not be able to duplicate documents even with read access."""
document = factories.DocumentFactory(link_reach="public")
response = APIClient().post(f"/api/v1.0/documents/{document.id!s}/duplicate/")
assert response.status_code == 401
assert models.Document.objects.count() == 1
@pytest.mark.parametrize("index", range(3))
def test_api_documents_duplicate_success(index):
"""
Anonymous users should be able to retrieve attachments linked to a public document.
Accesses should not be duplicated if the user does not request it specifically.
Attachments that are not in the content should not be passed for access in the
duplicated document's "attachments" list.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document_ids = [uuid.uuid4() for _ in range(3)]
image_refs = [get_image_refs(doc_id) for doc_id in document_ids]
# Create document content with the first image only
ydoc = pycrdt.Doc()
fragment = pycrdt.XmlFragment(
[
pycrdt.XmlElement("img", {"src": image_refs[0][1]}),
]
)
ydoc["document-store"] = fragment
update = ydoc.get_update()
base64_content = base64.b64encode(update).decode("utf-8")
# Create documents
document = factories.DocumentFactory(
id=document_ids[index],
content=base64_content,
link_reach="restricted",
users=[user, factories.UserFactory()],
title="document with an image",
attachments=[key for key, _ in image_refs],
)
factories.DocumentFactory(id=document_ids[(index + 1) % 3])
# Don't create document for third ID to check that it doesn't impact access to attachments
# Duplicate the document via the API endpoint
response = client.post(f"/api/v1.0/documents/{document.id}/duplicate/")
assert response.status_code == 201
duplicated_document = models.Document.objects.get(id=response.json()["id"])
assert duplicated_document.title == "Copy of document with an image"
assert duplicated_document.content == document.content
assert duplicated_document.creator == user
assert duplicated_document.link_reach == "restricted"
assert duplicated_document.link_role == "reader"
assert duplicated_document.duplicated_from == document
assert duplicated_document.attachments == [
image_refs[0][0]
] # Only the first image key
assert duplicated_document.get_parent() == document.get_parent()
assert duplicated_document.path == document.get_next_sibling().path
# Check that accesses were not duplicated.
# The user who did the duplicate is forced as owner
assert duplicated_document.accesses.count() == 1
access = duplicated_document.accesses.first()
assert access.user == user
assert access.role == "owner"
# Ensure access persists after the owner loses access to the original document
models.DocumentAccess.objects.filter(document=document).delete()
response = client.get(
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=image_refs[0][1]
)
assert response.status_code == 200
authorization = response["Authorization"]
assert "AWS4-HMAC-SHA256 Credential=" in authorization
assert (
"SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature="
in authorization
)
assert response["X-Amz-Date"] == timezone.now().strftime("%Y%m%dT%H%M%SZ")
s3_url = urlparse(settings.AWS_S3_ENDPOINT_URL)
response = requests.get(
f"{settings.AWS_S3_ENDPOINT_URL:s}/impress-media-storage/{image_refs[0][0]:s}",
headers={
"authorization": authorization,
"x-amz-date": response["x-amz-date"],
"x-amz-content-sha256": response["x-amz-content-sha256"],
"Host": f"{s3_url.hostname:s}:{s3_url.port:d}",
},
timeout=1,
)
assert response.content == PIXEL
# Ensure the other images are not accessible
for _, url in image_refs[1:]:
response = client.get(
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=url
)
assert response.status_code == 403
def test_api_documents_duplicate_with_accesses():
"""Accesses should be duplicated if the user requests it specifically."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(
users=[user],
title="document with accesses",
)
user_access = factories.UserDocumentAccessFactory(document=document)
team_access = factories.TeamDocumentAccessFactory(document=document)
# Duplicate the document via the API endpoint requesting to duplicate accesses
response = client.post(
f"/api/v1.0/documents/{document.id!s}/duplicate/",
{"with_accesses": True},
format="json",
)
assert response.status_code == 201
duplicated_document = models.Document.objects.get(id=response.json()["id"])
assert duplicated_document.title == "Copy of document with accesses"
assert duplicated_document.content == document.content
assert duplicated_document.link_reach == document.link_reach
assert duplicated_document.link_role == document.link_role
assert duplicated_document.creator == user
assert duplicated_document.duplicated_from == document
assert duplicated_document.attachments == []
# Check that accesses were duplicated and the user who did the duplicate is forced as owner
duplicated_accesses = duplicated_document.accesses
assert duplicated_accesses.count() == 3
assert duplicated_accesses.get(user=user).role == "owner"
assert duplicated_accesses.get(user=user_access.user).role == user_access.role
assert duplicated_accesses.get(team=team_access.team).role == team_access.role

View File

@@ -1,10 +1,10 @@
"""
Test file uploads API endpoint for users in impress's core app.
Test media-auth authorization API endpoint in docs core app.
"""
import uuid
from io import BytesIO
from urllib.parse import urlparse
from uuid import uuid4
from django.conf import settings
from django.core.files.storage import default_storage
@@ -14,19 +14,32 @@ import pytest
import requests
from rest_framework.test import APIClient
from core import factories
from core import factories, models
from core.tests.conftest import TEAM, USER, VIA
pytestmark = pytest.mark.django_db
def test_api_documents_media_auth_unkown_document():
"""
Trying to download a media related to a document ID that does not exist
should not have the side effect to create it (no regression test).
"""
original_url = f"http://localhost/media/{uuid4()!s}/attachments/{uuid4()!s}.jpg"
response = APIClient().get(
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=original_url
)
assert response.status_code == 403
assert models.Document.objects.exists() is False
def test_api_documents_media_auth_anonymous_public():
"""Anonymous users should be able to retrieve attachments linked to a public document"""
document = factories.DocumentFactory(link_reach="public")
filename = f"{uuid.uuid4()!s}.jpg"
key = f"{document.pk!s}/attachments/{filename:s}"
document_id = uuid4()
filename = f"{uuid4()!s}.jpg"
key = f"{document_id!s}/attachments/{filename:s}"
default_storage.connection.meta.client.put_object(
Bucket=default_storage.bucket_name,
Key=key,
@@ -34,6 +47,8 @@ def test_api_documents_media_auth_anonymous_public():
ContentType="text/plain",
)
factories.DocumentFactory(id=document_id, link_reach="public", attachments=[key])
original_url = f"http://localhost/media/{key:s}"
response = APIClient().get(
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=original_url
@@ -66,8 +81,6 @@ def test_api_documents_media_auth_anonymous_public():
def test_api_documents_media_auth_extensions():
"""Files with extensions of any format should work."""
document = factories.DocumentFactory(link_reach="public")
extensions = [
"c",
"go",
@@ -76,10 +89,15 @@ def test_api_documents_media_auth_extensions():
"woff2",
"appimage",
]
document_id = uuid4()
keys = []
for ext in extensions:
filename = f"{uuid.uuid4()!s}.{ext:s}"
key = f"{document.pk!s}/attachments/{filename:s}"
filename = f"{uuid4()!s}.{ext:s}"
keys.append(f"{document_id!s}/attachments/{filename:s}")
factories.DocumentFactory(link_reach="public", attachments=keys)
for key in keys:
original_url = f"http://localhost/media/{key:s}"
response = APIClient().get(
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=original_url
@@ -94,10 +112,11 @@ def test_api_documents_media_auth_anonymous_authenticated_or_restricted(reach):
Anonymous users should not be allowed to retrieve attachments linked to a document
with link reach set to authenticated or restricted.
"""
document = factories.DocumentFactory(link_reach=reach)
document_id = uuid4()
filename = f"{uuid4()!s}.jpg"
media_url = f"http://localhost/media/{document_id!s}/attachments/{filename:s}"
filename = f"{uuid.uuid4()!s}.jpg"
media_url = f"http://localhost/media/{document.pk!s}/attachments/{filename:s}"
factories.DocumentFactory(id=document_id, link_reach=reach)
response = APIClient().get(
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=media_url
@@ -107,20 +126,16 @@ def test_api_documents_media_auth_anonymous_authenticated_or_restricted(reach):
assert "Authorization" not in response
@pytest.mark.parametrize("reach", ["public", "authenticated"])
def test_api_documents_media_auth_authenticated_public_or_authenticated(reach):
def test_api_documents_media_auth_anonymous_attachments():
"""
Authenticated users who are not related to a document should be able to retrieve
attachments related to a document with public or authenticated link reach.
Declaring a media key as original attachment on a document to which
a user has access should give them access to the attachment file
regardless of their access rights on the original document.
"""
document = factories.DocumentFactory(link_reach=reach)
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
filename = f"{uuid.uuid4()!s}.jpg"
key = f"{document.pk!s}/attachments/{filename:s}"
document_id = uuid4()
filename = f"{uuid4()!s}.jpg"
key = f"{document_id!s}/attachments/{filename:s}"
media_url = f"http://localhost/media/{key:s}"
default_storage.connection.meta.client.put_object(
Bucket=default_storage.bucket_name,
@@ -129,9 +144,73 @@ def test_api_documents_media_auth_authenticated_public_or_authenticated(reach):
ContentType="text/plain",
)
original_url = f"http://localhost/media/{key:s}"
factories.DocumentFactory(id=document_id, link_reach="restricted")
response = APIClient().get(
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=media_url
)
assert response.status_code == 403
# Let's now add a document to which the anonymous user has access and
# pointing to the attachment
parent = factories.DocumentFactory(link_reach="public")
factories.DocumentFactory(parent=parent, link_reach="restricted", attachments=[key])
response = APIClient().get(
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=media_url
)
assert response.status_code == 200
authorization = response["Authorization"]
assert "AWS4-HMAC-SHA256 Credential=" in authorization
assert (
"SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature="
in authorization
)
assert response["X-Amz-Date"] == timezone.now().strftime("%Y%m%dT%H%M%SZ")
s3_url = urlparse(settings.AWS_S3_ENDPOINT_URL)
file_url = f"{settings.AWS_S3_ENDPOINT_URL:s}/impress-media-storage/{key:s}"
response = requests.get(
file_url,
headers={
"authorization": authorization,
"x-amz-date": response["x-amz-date"],
"x-amz-content-sha256": response["x-amz-content-sha256"],
"Host": f"{s3_url.hostname:s}:{s3_url.port:d}",
},
timeout=1,
)
assert response.content.decode("utf-8") == "my prose"
@pytest.mark.parametrize("reach", ["public", "authenticated"])
def test_api_documents_media_auth_authenticated_public_or_authenticated(reach):
"""
Authenticated users who are not related to a document should be able to retrieve
attachments related to a document with public or authenticated link reach.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document_id = uuid4()
filename = f"{uuid4()!s}.jpg"
key = f"{document_id!s}/attachments/{filename:s}"
media_url = f"http://localhost/media/{key:s}"
default_storage.connection.meta.client.put_object(
Bucket=default_storage.bucket_name,
Key=key,
Body=BytesIO(b"my prose"),
ContentType="text/plain",
)
factories.DocumentFactory(id=document_id, link_reach=reach, attachments=[key])
response = client.get(
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=original_url
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=media_url
)
assert response.status_code == 200
@@ -164,14 +243,18 @@ def test_api_documents_media_auth_authenticated_restricted():
Authenticated users who are not related to a document should not be allowed to
retrieve attachments linked to a document that is restricted.
"""
document = factories.DocumentFactory(link_reach="restricted")
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
filename = f"{uuid.uuid4()!s}.jpg"
media_url = f"http://localhost/media/{document.pk!s}/attachments/{filename:s}"
document_id = uuid4()
filename = f"{uuid4()!s}.jpg"
key = f"{document_id!s}/attachments/{filename:s}"
media_url = f"http://localhost/media/{key:s}"
factories.DocumentFactory(
id=document_id, link_reach="restricted", attachments=[key]
)
response = client.get(
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=media_url
@@ -191,16 +274,10 @@ def test_api_documents_media_auth_related(via, mock_user_teams):
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory()
if via == USER:
factories.UserDocumentAccessFactory(document=document, user=user)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamDocumentAccessFactory(document=document, team="lasuite")
filename = f"{uuid.uuid4()!s}.jpg"
key = f"{document.pk!s}/attachments/{filename:s}"
document_id = uuid4()
filename = f"{uuid4()!s}.jpg"
key = f"{document_id!s}/attachments/{filename:s}"
media_url = f"http://localhost/media/{key:s}"
default_storage.connection.meta.client.put_object(
Bucket=default_storage.bucket_name,
Key=key,
@@ -208,9 +285,17 @@ def test_api_documents_media_auth_related(via, mock_user_teams):
ContentType="text/plain",
)
original_url = f"http://localhost/media/{key:s}"
document = factories.DocumentFactory(
id=document_id, link_reach="restricted", attachments=[key]
)
if via == USER:
factories.UserDocumentAccessFactory(document=document, user=user)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamDocumentAccessFactory(document=document, team="lasuite")
response = client.get(
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=original_url
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=media_url
)
assert response.status_code == 200

View File

@@ -37,6 +37,7 @@ def test_api_documents_retrieve_anonymous_public_standalone():
"cors_proxy": True,
"descendants": True,
"destroy": False,
"duplicate": True,
# Anonymous user can't favorite a document even with read access
"favorite": False,
"invite_owner": False,
@@ -103,6 +104,7 @@ def test_api_documents_retrieve_anonymous_public_parent():
"descendants": True,
"cors_proxy": True,
"destroy": False,
"duplicate": True,
# Anonymous user can't favorite a document even with read access
"favorite": False,
"invite_owner": False,
@@ -198,6 +200,7 @@ def test_api_documents_retrieve_authenticated_unrelated_public_or_authenticated(
"descendants": True,
"cors_proxy": True,
"destroy": False,
"duplicate": True,
"favorite": True,
"invite_owner": False,
"link_configuration": False,
@@ -271,6 +274,7 @@ def test_api_documents_retrieve_authenticated_public_or_authenticated_parent(rea
"descendants": True,
"cors_proxy": True,
"destroy": False,
"duplicate": True,
"favorite": True,
"invite_owner": False,
"link_configuration": False,
@@ -450,6 +454,7 @@ def test_api_documents_retrieve_authenticated_related_parent():
"descendants": True,
"cors_proxy": True,
"destroy": access.role == "owner",
"duplicate": True,
"favorite": True,
"invite_owner": access.role == "owner",
"link_configuration": access.role in ["administrator", "owner"],
@@ -784,7 +789,7 @@ def test_api_documents_retrieve_user_roles(django_assert_max_num_queries):
)
expected_roles = {access.role for access in accesses}
with django_assert_max_num_queries(12):
with django_assert_max_num_queries(14):
response = client.get(f"/api/v1.0/documents/{document.id!s}/")
assert response.status_code == 200

View File

@@ -81,6 +81,7 @@ def test_api_documents_trashbin_format():
"descendants": True,
"cors_proxy": True,
"destroy": True,
"duplicate": True,
"favorite": True,
"invite_owner": True,
"link_configuration": True,

View File

@@ -328,3 +328,22 @@ def test_api_documents_update_administrator_or_owner_of_another(via, mock_user_t
other_document.refresh_from_db()
other_document_values = serializers.DocumentSerializer(instance=other_document).data
assert other_document_values == old_document_values
def test_api_documents_update_invalid_content():
"""
Updating a document with a non base64 encoded content should raise a validation error.
"""
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(users=[[user, "owner"]])
response = client.put(
f"/api/v1.0/documents/{document.id!s}/",
{"content": "invalid content"},
format="json",
)
assert response.status_code == 400
assert response.json() == {"content": ["Invalid base64 content."]}

View File

@@ -0,0 +1,154 @@
"""
Test extract-attachments on document update in docs core app.
"""
import base64
from uuid import uuid4
import pycrdt
import pytest
from rest_framework.test import APIClient
from core import factories
pytestmark = pytest.mark.django_db
def get_ydoc_with_mages(image_keys):
"""Return a ydoc from text for testing purposes."""
ydoc = pycrdt.Doc()
fragment = pycrdt.XmlFragment(
[
pycrdt.XmlElement("img", {"src": f"http://localhost/media/{key:s}"})
for key in image_keys
]
)
ydoc["document-store"] = fragment
update = ydoc.get_update()
return base64.b64encode(update).decode("utf-8")
def test_api_documents_update_new_attachment_keys_anonymous(django_assert_num_queries):
"""
When an anonymous user updates a document, the attachment keys extracted from the
updated content should be added to the list of "attachments" ot the document if these
attachments are already readable by anonymous users.
"""
image_keys = [f"{uuid4()!s}/attachments/{uuid4()!s}.png" for _ in range(4)]
document = factories.DocumentFactory(
content=get_ydoc_with_mages(image_keys[:1]),
attachments=[image_keys[0]],
link_reach="public",
link_role="editor",
)
factories.DocumentFactory(attachments=[image_keys[1]], link_reach="public")
factories.DocumentFactory(attachments=[image_keys[2]], link_reach="authenticated")
factories.DocumentFactory(attachments=[image_keys[3]], link_reach="restricted")
expected_keys = {image_keys[i] for i in [0, 1]}
with django_assert_num_queries(9):
response = APIClient().put(
f"/api/v1.0/documents/{document.id!s}/",
{"content": get_ydoc_with_mages(image_keys)},
format="json",
)
assert response.status_code == 200
document.refresh_from_db()
assert set(document.attachments) == expected_keys
# Check that the db query to check attachments readability for extracted
# keys is not done if the content changes but no new keys are found
with django_assert_num_queries(5):
response = APIClient().put(
f"/api/v1.0/documents/{document.id!s}/",
{"content": get_ydoc_with_mages(image_keys[:2])},
format="json",
)
assert response.status_code == 200
document.refresh_from_db()
assert len(document.attachments) == 2
assert set(document.attachments) == expected_keys
def test_api_documents_update_new_attachment_keys_authenticated(
django_assert_num_queries,
):
"""
When an authenticated user updates a document, the attachment keys extracted from the
updated content should be added to the list of "attachments" ot the document if these
attachments are already readable by the editing user.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
image_keys = [f"{uuid4()!s}/attachments/{uuid4()!s}.png" for _ in range(5)]
document = factories.DocumentFactory(
content=get_ydoc_with_mages(image_keys[:1]),
attachments=[image_keys[0]],
users=[(user, "editor")],
)
factories.DocumentFactory(attachments=[image_keys[1]], link_reach="public")
factories.DocumentFactory(attachments=[image_keys[2]], link_reach="authenticated")
factories.DocumentFactory(attachments=[image_keys[3]], link_reach="restricted")
factories.DocumentFactory(attachments=[image_keys[4]], users=[user])
expected_keys = {image_keys[i] for i in [0, 1, 2, 4]}
with django_assert_num_queries(10):
response = client.put(
f"/api/v1.0/documents/{document.id!s}/",
{"content": get_ydoc_with_mages(image_keys)},
format="json",
)
assert response.status_code == 200
document.refresh_from_db()
assert set(document.attachments) == expected_keys
# Check that the db query to check attachments readability for extracted
# keys is not done if the content changes but no new keys are found
with django_assert_num_queries(6):
response = client.put(
f"/api/v1.0/documents/{document.id!s}/",
{"content": get_ydoc_with_mages(image_keys[:2])},
format="json",
)
assert response.status_code == 200
document.refresh_from_db()
assert len(document.attachments) == 4
assert set(document.attachments) == expected_keys
def test_api_documents_update_new_attachment_keys_duplicate():
"""
Duplicate keys in the content should not result in duplicates in the document's attachments.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
image_key1 = f"{uuid4()!s}/attachments/{uuid4()!s}.png"
image_key2 = f"{uuid4()!s}/attachments/{uuid4()!s}.png"
document = factories.DocumentFactory(
content=get_ydoc_with_mages([image_key1]),
attachments=[image_key1],
users=[(user, "editor")],
)
factories.DocumentFactory(attachments=[image_key2], users=[user])
response = client.put(
f"/api/v1.0/documents/{document.id!s}/",
{"content": get_ydoc_with_mages([image_key1, image_key2, image_key2])},
format="json",
)
assert response.status_code == 200
document.refresh_from_db()
assert len(document.attachments) == 2
assert set(document.attachments) == {image_key1, image_key2}

View File

@@ -1,35 +0,0 @@
import pytest
from core import factories
@pytest.mark.django_db
def test_update_blank_title_migration(migrator):
"""
Test that the migration fixes the titles of documents that are
"Untitled document", "Unbenanntes Dokument" or "Document sans titre"
"""
migrator.apply_initial_migration(("core", "0017_add_fields_for_soft_delete"))
english_doc = factories.DocumentFactory(title="Untitled document")
german_doc = factories.DocumentFactory(title="Unbenanntes Dokument")
french_doc = factories.DocumentFactory(title="Document sans titre")
other_doc = factories.DocumentFactory(title="My document")
assert english_doc.title == "Untitled document"
assert german_doc.title == "Unbenanntes Dokument"
assert french_doc.title == "Document sans titre"
assert other_doc.title == "My document"
# Apply the migration
migrator.apply_tested_migration(("core", "0018_update_blank_title"))
english_doc.refresh_from_db()
german_doc.refresh_from_db()
french_doc.refresh_from_db()
other_doc.refresh_from_db()
assert english_doc.title == None
assert german_doc.title == None
assert french_doc.title == None
assert other_doc.title == "My document"

View File

@@ -0,0 +1,47 @@
import pytest
from core import models
@pytest.mark.django_db
def test_update_blank_title_migration(migrator):
"""
Test that the migration fixes the titles of documents that are
"Untitled document", "Unbenanntes Dokument" or "Document sans titre"
"""
old_state = migrator.apply_initial_migration(
("core", "0017_add_fields_for_soft_delete")
)
OldDocument = old_state.apps.get_model("core", "Document")
old_english_doc = OldDocument.objects.create(
title="Untitled document", depth=1, path="0000001"
)
old_german_doc = OldDocument.objects.create(
title="Unbenanntes Dokument", depth=1, path="0000002"
)
old_french_doc = OldDocument.objects.create(
title="Document sans titre", depth=1, path="0000003"
)
old_other_doc = OldDocument.objects.create(
title="My document", depth=1, path="0000004"
)
assert old_english_doc.title == "Untitled document"
assert old_german_doc.title == "Unbenanntes Dokument"
assert old_french_doc.title == "Document sans titre"
assert old_other_doc.title == "My document"
# Apply the migration
new_state = migrator.apply_tested_migration(("core", "0018_update_blank_title"))
NewDocument = new_state.apps.get_model("core", "Document")
new_english_doc = NewDocument.objects.get(pk=old_english_doc.pk)
new_german_doc = NewDocument.objects.get(pk=old_german_doc.pk)
new_french_doc = NewDocument.objects.get(pk=old_french_doc.pk)
new_other_doc = NewDocument.objects.get(pk=old_other_doc.pk)
assert new_english_doc.title == None
assert new_german_doc.title == None
assert new_french_doc.title == None
assert new_other_doc.title == "My document"

View File

@@ -0,0 +1,54 @@
import base64
import uuid
from django.core.files.base import ContentFile
from django.core.files.storage import default_storage
import pycrdt
import pytest
from core import models
@pytest.mark.django_db
def test_populate_attachments_on_all_documents(migrator):
"""Test that the migration populates attachments on existing documents."""
old_state = migrator.apply_initial_migration(
("core", "0019_alter_user_language_default_to_null")
)
OldDocument = old_state.apps.get_model("core", "Document")
old_doc_without_attachments = OldDocument.objects.create(
title="Doc without attachments", depth=1, path="0000002"
)
old_doc_with_attachments = OldDocument.objects.create(
title="Doc with attachments", depth=1, path="0000001"
)
# Create document content with an image
file_key = f"{old_doc_with_attachments.id!s}/file"
image_key = f"{old_doc_with_attachments.id!s}/attachments/{uuid.uuid4()!s}.png"
ydoc = pycrdt.Doc()
fragment = pycrdt.XmlFragment(
[pycrdt.XmlElement("img", {"src": f"http://localhost/media/{image_key:s}"})]
)
ydoc["document-store"] = fragment
update = ydoc.get_update()
base64_content = base64.b64encode(update).decode("utf-8")
bytes_content = base64_content.encode("utf-8")
content_file = ContentFile(bytes_content)
default_storage.save(file_key, content_file)
# Apply the migration
new_state = migrator.apply_tested_migration(
("core", "0020_remove_is_public_add_field_attachments_and_duplicated_from")
)
NewDocument = new_state.apps.get_model("core", "Document")
new_doc_with_attachments = NewDocument.objects.get(pk=old_doc_with_attachments.pk)
new_doc_without_attachments = NewDocument.objects.get(
pk=old_doc_without_attachments.pk
)
assert new_doc_without_attachments.attachments == []
assert new_doc_with_attachments.attachments == [image_key]

View File

@@ -18,6 +18,9 @@ pytestmark = pytest.mark.django_db
@override_settings(
COLLABORATION_WS_URL="http://testcollab/",
CRISP_WEBSITE_ID="123",
FRONTEND_CSS_URL="http://testcss/",
FRONTEND_HOMEPAGE_FEATURE_ENABLED=True,
FRONTEND_FOOTER_FEATURE_ENABLED=True,
FRONTEND_THEME="test-theme",
MEDIA_BASE_URL="http://testserver/",
POSTHOG_KEY={"id": "132456", "host": "https://eu.i.posthog-test.com"},
@@ -38,6 +41,9 @@ def test_api_config(is_authenticated):
"COLLABORATION_WS_URL": "http://testcollab/",
"CRISP_WEBSITE_ID": "123",
"ENVIRONMENT": "test",
"FRONTEND_CSS_URL": "http://testcss/",
"FRONTEND_HOMEPAGE_FEATURE_ENABLED": True,
"FRONTEND_FOOTER_FEATURE_ENABLED": True,
"FRONTEND_THEME": "test-theme",
"LANGUAGES": [
["en-us", "English"],
@@ -49,4 +55,5 @@ def test_api_config(is_authenticated):
"MEDIA_BASE_URL": "http://testserver/",
"POSTHOG_KEY": {"id": "132456", "host": "https://eu.i.posthog-test.com"},
"SENTRY_DSN": "https://sentry.test/123",
"AI_FEATURE_ENABLED": False,
}

View File

@@ -0,0 +1,81 @@
"""Test the footer API."""
import responses
from rest_framework.test import APIClient
def test_api_footer_without_settings_configured(settings):
"""Test the footer API without settings configured."""
settings.FRONTEND_URL_JSON_FOOTER = None
client = APIClient()
response = client.get("/api/v1.0/footer/")
assert response.status_code == 200
assert response.json() == {}
@responses.activate
def test_api_footer_with_invalid_request(settings):
"""Test the footer API with an invalid request."""
settings.FRONTEND_URL_JSON_FOOTER = "https://invalid-request.com"
footer_response = responses.get(settings.FRONTEND_URL_JSON_FOOTER, status=404)
client = APIClient()
response = client.get("/api/v1.0/footer/")
assert response.status_code == 200
assert response.json() == {}
assert footer_response.call_count == 1
@responses.activate
def test_api_footer_with_invalid_json(settings):
"""Test the footer API with an invalid JSON response."""
settings.FRONTEND_URL_JSON_FOOTER = "https://valid-request.com"
footer_response = responses.get(
settings.FRONTEND_URL_JSON_FOOTER, status=200, body="invalid json"
)
client = APIClient()
response = client.get("/api/v1.0/footer/")
assert response.status_code == 200
assert response.json() == {}
assert footer_response.call_count == 1
@responses.activate
def test_api_footer_with_valid_json(settings):
"""Test the footer API with an invalid JSON response."""
settings.FRONTEND_URL_JSON_FOOTER = "https://valid-request.com"
footer_response = responses.get(
settings.FRONTEND_URL_JSON_FOOTER, status=200, json={"foo": "bar"}
)
client = APIClient()
response = client.get("/api/v1.0/footer/")
assert response.status_code == 200
assert response.json() == {"foo": "bar"}
assert footer_response.call_count == 1
@responses.activate
def test_api_footer_with_valid_json_and_cache(settings):
"""Test the footer API with an invalid JSON response."""
settings.FRONTEND_URL_JSON_FOOTER = "https://valid-request.com"
footer_response = responses.get(
settings.FRONTEND_URL_JSON_FOOTER, status=200, json={"foo": "bar"}
)
client = APIClient()
response = client.get("/api/v1.0/footer/")
assert response.status_code == 200
assert response.json() == {"foo": "bar"}
assert footer_response.call_count == 1
response = client.get("/api/v1.0/footer/")
assert response.status_code == 200
assert response.json() == {"foo": "bar"}
# The cache should have been used
assert footer_response.call_count == 1

View File

@@ -4,10 +4,8 @@ Test throttling on documents for the AI endpoint.
from unittest.mock import patch
from django.core.cache import cache
from django.test import override_settings
import pytest
from rest_framework.response import Response
from rest_framework.test import APIRequestFactory
from rest_framework.views import APIView
@@ -25,12 +23,6 @@ class DocumentAPIView(APIView):
return Response({"message": "Success"})
@pytest.fixture(autouse=True)
def clear_cache():
"""Fixture to clear the cache before each test."""
cache.clear()
@override_settings(AI_DOCUMENT_RATE_THROTTLE_RATES={"minute": 3, "hour": 6, "day": 10})
@patch("time.time")
def test_api_utils_ai_document_rate_throttle_minute_limit(mock_time):

View File

@@ -5,7 +5,6 @@ Test throttling on users for the AI endpoint.
from unittest.mock import patch
from uuid import uuid4
from django.core.cache import cache
from django.test import override_settings
import pytest
@@ -29,12 +28,6 @@ class DocumentAPIView(APIView):
return Response({"message": "Success"})
@pytest.fixture(autouse=True)
def clear_cache():
"""Fixture to clear the cache before each test."""
cache.clear()
@override_settings(AI_USER_RATE_THROTTLE_RATES={"minute": 3, "hour": 6, "day": 10})
@patch("time.time")
def test_api_utils_ai_user_rate_throttle_minute_limit(mock_time):

View File

@@ -7,7 +7,7 @@ from django.core.exceptions import ValidationError
import pytest
from core import factories
from core import factories, models
pytestmark = pytest.mark.django_db
@@ -294,7 +294,7 @@ def test_models_document_access_get_abilities_for_editor_of_owner():
abilities = access.get_abilities(user)
assert abilities == {
"destroy": False,
"retrieve": True,
"retrieve": False,
"update": False,
"partial_update": False,
"set_role_to": [],
@@ -311,7 +311,7 @@ def test_models_document_access_get_abilities_for_editor_of_administrator():
abilities = access.get_abilities(user)
assert abilities == {
"destroy": False,
"retrieve": True,
"retrieve": False,
"update": False,
"partial_update": False,
"set_role_to": [],
@@ -333,7 +333,7 @@ def test_models_document_access_get_abilities_for_editor_of_editor_user(
assert abilities == {
"destroy": False,
"retrieve": True,
"retrieve": False,
"update": False,
"partial_update": False,
"set_role_to": [],
@@ -353,7 +353,7 @@ def test_models_document_access_get_abilities_for_reader_of_owner():
abilities = access.get_abilities(user)
assert abilities == {
"destroy": False,
"retrieve": True,
"retrieve": False,
"update": False,
"partial_update": False,
"set_role_to": [],
@@ -370,7 +370,7 @@ def test_models_document_access_get_abilities_for_reader_of_administrator():
abilities = access.get_abilities(user)
assert abilities == {
"destroy": False,
"retrieve": True,
"retrieve": False,
"update": False,
"partial_update": False,
"set_role_to": [],
@@ -392,7 +392,7 @@ def test_models_document_access_get_abilities_for_reader_of_reader_user(
assert abilities == {
"destroy": False,
"retrieve": True,
"retrieve": False,
"update": False,
"partial_update": False,
"set_role_to": [],
@@ -412,8 +412,16 @@ def test_models_document_access_get_abilities_preset_role(django_assert_num_quer
assert abilities == {
"destroy": False,
"retrieve": True,
"retrieve": False,
"update": False,
"partial_update": False,
"set_role_to": [],
}
@pytest.mark.parametrize("role", models.RoleChoices)
def test_models_document_access_get_abilities_retrieve_own_access(role):
"""Check abilities of self access for the owner of a document."""
access = factories.UserDocumentAccessFactory(role=role)
abilities = access.get_abilities(access.user)
assert abilities["retrieve"] is True

View File

@@ -161,6 +161,7 @@ def test_models_documents_get_abilities_forbidden(
"descendants": False,
"cors_proxy": False,
"destroy": False,
"duplicate": False,
"favorite": False,
"invite_owner": False,
"media_auth": False,
@@ -220,6 +221,7 @@ def test_models_documents_get_abilities_reader(
"descendants": True,
"cors_proxy": True,
"destroy": False,
"duplicate": True,
"favorite": is_authenticated,
"invite_owner": False,
"link_configuration": False,
@@ -281,6 +283,7 @@ def test_models_documents_get_abilities_editor(
"descendants": True,
"cors_proxy": True,
"destroy": False,
"duplicate": True,
"favorite": is_authenticated,
"invite_owner": False,
"link_configuration": False,
@@ -331,6 +334,7 @@ def test_models_documents_get_abilities_owner(django_assert_num_queries):
"descendants": True,
"cors_proxy": True,
"destroy": True,
"duplicate": True,
"favorite": True,
"invite_owner": True,
"link_configuration": True,
@@ -378,6 +382,7 @@ def test_models_documents_get_abilities_administrator(django_assert_num_queries)
"descendants": True,
"cors_proxy": True,
"destroy": False,
"duplicate": True,
"favorite": True,
"invite_owner": False,
"link_configuration": True,
@@ -428,6 +433,7 @@ def test_models_documents_get_abilities_editor_user(django_assert_num_queries):
"descendants": True,
"cors_proxy": True,
"destroy": False,
"duplicate": True,
"favorite": True,
"invite_owner": False,
"link_configuration": False,
@@ -485,6 +491,7 @@ def test_models_documents_get_abilities_reader_user(
"descendants": True,
"cors_proxy": True,
"destroy": False,
"duplicate": True,
"favorite": True,
"invite_owner": False,
"link_configuration": False,
@@ -540,6 +547,7 @@ def test_models_documents_get_abilities_preset_role(django_assert_num_queries):
"descendants": True,
"cors_proxy": True,
"destroy": False,
"duplicate": True,
"favorite": True,
"invite_owner": False,
"link_configuration": False,
@@ -1297,3 +1305,47 @@ def test_models_documents_restore_complex_bis(django_assert_num_queries):
def test_models_documents_get_select_options(ancestors_links, select_options):
"""Validate that the "get_select_options" method operates as expected."""
assert models.LinkReachChoices.get_select_options(ancestors_links) == select_options
def test_models_documents_compute_ancestors_links_no_highest_readable():
"""Test the compute_ancestors_links method."""
document = factories.DocumentFactory(link_reach="public")
assert document.compute_ancestors_links(user=AnonymousUser()) == []
def test_models_documents_compute_ancestors_links_highest_readable(
django_assert_num_queries,
):
"""Test the compute_ancestors_links method."""
user = factories.UserFactory()
other_user = factories.UserFactory()
root = factories.DocumentFactory(
link_reach="restricted", link_role="reader", users=[user]
)
factories.DocumentFactory(
parent=root, link_reach="public", link_role="reader", users=[user]
)
child2 = factories.DocumentFactory(
parent=root,
link_reach="authenticated",
link_role="editor",
users=[user, other_user],
)
child3 = factories.DocumentFactory(
parent=child2,
link_reach="authenticated",
link_role="reader",
users=[user, other_user],
)
with django_assert_num_queries(2):
assert child3.compute_ancestors_links(user=user) == [
{"link_reach": root.link_reach, "link_role": root.link_role},
{"link_reach": child2.link_reach, "link_role": child2.link_role},
]
with django_assert_num_queries(2):
assert child3.compute_ancestors_links(user=other_user) == [
{"link_reach": child2.link_reach, "link_role": child2.link_role},
]

View File

@@ -0,0 +1,77 @@
"""Test util base64_yjs_to_text."""
import base64
import uuid
import pycrdt
from core import utils
# This base64 string is an example of what is saved in the database.
# This base64 is generated from the blocknote editor, it contains
# the text \n# *Hello* \n- w**or**ld
TEST_BASE64_STRING = (
"AR717vLVDgAHAQ5kb2N1bWVudC1zdG9yZQMKYmxvY2tHcm91cAcA9e7y1Q4AAw5ibG9ja0NvbnRh"
"aW5lcgcA9e7y1Q4BAwdoZWFkaW5nBwD17vLVDgIGBgD17vLVDgMGaXRhbGljAnt9hPXu8tUOBAVI"
"ZWxsb4b17vLVDgkGaXRhbGljBG51bGwoAPXu8tUOAg10ZXh0QWxpZ25tZW50AXcEbGVmdCgA9e7y"
"1Q4CBWxldmVsAX0BKAD17vLVDgECaWQBdyQwNGQ2MjM0MS04MzI2LTQyMzYtYTA4My00ODdlMjZm"
"YWQyMzAoAPXu8tUOAQl0ZXh0Q29sb3IBdwdkZWZhdWx0KAD17vLVDgEPYmFja2dyb3VuZENvbG9y"
"AXcHZGVmYXVsdIf17vLVDgEDDmJsb2NrQ29udGFpbmVyBwD17vLVDhADDmJ1bGxldExpc3RJdGVt"
"BwD17vLVDhEGBAD17vLVDhIBd4b17vLVDhMEYm9sZAJ7fYT17vLVDhQCb3KG9e7y1Q4WBGJvbGQE"
"bnVsbIT17vLVDhcCbGQoAPXu8tUOEQ10ZXh0QWxpZ25tZW50AXcEbGVmdCgA9e7y1Q4QAmlkAXck"
"ZDM1MWUwNjgtM2U1NS00MjI2LThlYTUtYWJiMjYzMTk4ZTJhKAD17vLVDhAJdGV4dENvbG9yAXcH"
"ZGVmYXVsdCgA9e7y1Q4QD2JhY2tncm91bmRDb2xvcgF3B2RlZmF1bHSH9e7y1Q4QAw5ibG9ja0Nv"
"bnRhaW5lcgcA9e7y1Q4eAwlwYXJhZ3JhcGgoAPXu8tUOHw10ZXh0QWxpZ25tZW50AXcEbGVmdCgA"
"9e7y1Q4eAmlkAXckODk3MDBjMDctZTBlMS00ZmUwLWFjYTItODQ5MzIwOWE3ZTQyKAD17vLVDh4J"
"dGV4dENvbG9yAXcHZGVmYXVsdCgA9e7y1Q4eD2JhY2tncm91bmRDb2xvcgF3B2RlZmF1bHQA"
)
def test_utils_base64_yjs_to_text():
"""Test extract text from saved yjs document"""
assert utils.base64_yjs_to_text(TEST_BASE64_STRING) == "Hello w or ld"
def test_utils_base64_yjs_to_xml():
"""Test extract xml from saved yjs document"""
content = utils.base64_yjs_to_xml(TEST_BASE64_STRING)
assert (
'<heading textAlignment="left" level="1"><italic>Hello</italic></heading>'
in content
or '<heading level="1" textAlignment="left"><italic>Hello</italic></heading>'
in content
)
assert (
'<bulletListItem textAlignment="left">w<bold>or</bold>ld</bulletListItem>'
in content
)
def test_utils_extract_attachments():
"""
All attachment keys in the document content should be extracted.
"""
document_id = uuid.uuid4()
image_key1 = f"{document_id!s}/attachments/{uuid.uuid4()!s}.png"
image_url1 = f"http://localhost/media/{image_key1:s}"
image_key2 = f"{uuid.uuid4()!s}/attachments/{uuid.uuid4()!s}.png"
image_url2 = f"http://localhost/{image_key2:s}"
image_key3 = f"{uuid.uuid4()!s}/attachments/{uuid.uuid4()!s}.png"
image_url3 = f"http://localhost/media/{image_key3:s}"
ydoc = pycrdt.Doc()
frag = pycrdt.XmlFragment(
[
pycrdt.XmlElement("img", {"src": image_url1}),
pycrdt.XmlElement("img", {"src": image_url2}),
pycrdt.XmlElement("p", {}, [pycrdt.XmlText(image_url3)]),
]
)
ydoc["document-store"] = frag
update = ydoc.get_update()
base64_string = base64.b64encode(update).decode("utf-8")
# image_key2 is missing the "/media/" part and shouldn't get extracted
assert utils.extract_attachments(base64_string) == [image_key1, image_key3]

View File

@@ -0,0 +1,163 @@
"""
Unit tests for the filter_root_paths utility function.
"""
from core.utils import filter_descendants
def test_utils_filter_descendants_success():
"""
The `filter_descendants` function should correctly identify descendant paths
from a given list of paths and root paths.
This test verifies that the function returns only the paths that have a prefix
matching one of the root paths.
"""
paths = [
"0001",
"00010001",
"000100010001",
"000100010002",
"000100020001",
"000100020002",
"0002",
"00020001",
"00020002",
"00030001",
"000300010001",
"00030002",
"0004",
"000400010003",
"0004000100030001",
"000400010004",
]
root_paths = [
"0001",
"0002",
"000400010003",
]
filtered_paths = filter_descendants(paths, root_paths, skip_sorting=True)
assert filtered_paths == [
"0001",
"00010001",
"000100010001",
"000100010002",
"000100020001",
"000100020002",
"0002",
"00020001",
"00020002",
"000400010003",
"0004000100030001",
]
def test_utils_filter_descendants_sorting():
"""
The `filter_descendants` function should handle unsorted input when sorting is enabled.
This test verifies that the function sorts the input if sorting is not skipped
and still correctly identifies accessible descendant paths.
"""
paths = [
"000300010001",
"000100010002",
"0001",
"00010001",
"000100010001",
"000100020002",
"000100020001",
"0002",
"00020001",
"00020002",
"00030001",
"00030002",
"0004000100030001",
"0004",
"000400010003",
"000400010004",
]
root_paths = [
"0002",
"000400010003",
"0001",
]
filtered_paths = filter_descendants(paths, root_paths)
assert filtered_paths == [
"0001",
"00010001",
"000100010001",
"000100010002",
"000100020001",
"000100020002",
"0002",
"00020001",
"00020002",
"000400010003",
"0004000100030001",
]
filtered_paths = filter_descendants(paths, root_paths, skip_sorting=True)
assert filtered_paths == [
"0001",
"00010001",
"000100010001",
"000100010002",
"000100020001",
"000100020002",
"0002",
"00020001",
"00020002",
"000400010003",
"0004000100030001",
]
def test_utils_filter_descendants_empty():
"""
The function should return an empty list if one or both inputs are empty.
"""
assert not filter_descendants([], ["0001"])
assert not filter_descendants(["0001"], [])
assert not filter_descendants([], [])
def test_utils_filter_descendants_no_match():
"""
The function should return an empty list if no path starts with any root path.
"""
paths = ["0001", "0002", "0003"]
root_paths = ["0004", "0005"]
assert not filter_descendants(paths, root_paths, skip_sorting=True)
def test_utils_filter_descendants_exact_match():
"""
The function should include paths that exactly match a root path.
"""
paths = ["0001", "0002", "0003"]
root_paths = ["0001", "0002"]
assert filter_descendants(paths, root_paths, skip_sorting=True) == ["0001", "0002"]
def test_utils_filter_descendants_single_root_matches_all():
"""
A single root path should match all its descendants.
"""
paths = ["0001", "00010001", "000100010001", "00010002"]
root_paths = ["0001"]
assert filter_descendants(paths, root_paths) == [
"0001",
"00010001",
"000100010001",
"00010002",
]
def test_utils_filter_descendants_path_shorter_than_root():
"""
A path shorter than any root path should not match.
"""
paths = ["0001", "0002"]
root_paths = ["00010001"]
assert not filter_descendants(paths, root_paths)

View File

@@ -56,4 +56,5 @@ urlpatterns = [
),
),
path(f"api/{settings.API_VERSION}/config/", viewsets.ConfigView.as_view()),
path(f"api/{settings.API_VERSION}/footer/", viewsets.FooterView.as_view()),
]

76
src/backend/core/utils.py Normal file
View File

@@ -0,0 +1,76 @@
"""Utils for the core app."""
import base64
import re
import pycrdt
from bs4 import BeautifulSoup
from core import enums
def filter_descendants(paths, root_paths, skip_sorting=False):
"""
Filters paths to keep only those that are descendants of any path in root_paths.
A path is considered a descendant of a root path if it starts with the root path.
If `skip_sorting` is not set to True, the function will sort both lists before
processing because both `paths` and `root_paths` need to be in lexicographic order
before going through the algorithm.
Args:
paths (iterable of str): List of paths to be filtered.
root_paths (iterable of str): List of paths to check as potential prefixes.
skip_sorting (bool): If True, assumes both `paths` and `root_paths` are already sorted.
Returns:
list of str: A list of sorted paths that are descendants of any path in `root_paths`.
"""
results = []
i = 0
n = len(root_paths)
if not skip_sorting:
paths.sort()
root_paths.sort()
for path in paths:
# Try to find a matching prefix in the sorted accessible paths
while i < n:
if path.startswith(root_paths[i]):
results.append(path)
break
if root_paths[i] < path:
i += 1
else:
# If paths[i] > path, no need to keep searching
break
return results
def base64_yjs_to_xml(base64_string):
"""Extract xml from base64 yjs document."""
decoded_bytes = base64.b64decode(base64_string)
# uint8_array = bytearray(decoded_bytes)
doc = pycrdt.Doc()
doc.apply_update(decoded_bytes)
return str(doc.get("document-store", type=pycrdt.XmlFragment))
def base64_yjs_to_text(base64_string):
"""Extract text from base64 yjs document."""
blocknote_structure = base64_yjs_to_xml(base64_string)
soup = BeautifulSoup(blocknote_structure, "lxml-xml")
return soup.get_text(separator=" ", strip=True)
def extract_attachments(content):
"""Helper method to extract media paths from a document's content."""
if not content:
return []
xml_content = base64_yjs_to_xml(content)
return re.findall(enums.MEDIA_STORAGE_URL_EXTRACT, xml_content)

View File

@@ -410,6 +410,27 @@ class Base(Configuration):
FRONTEND_THEME = values.Value(
None, environ_name="FRONTEND_THEME", environ_prefix=None
)
FRONTEND_HOMEPAGE_FEATURE_ENABLED = values.BooleanValue(
default=False,
environ_name="FRONTEND_HOMEPAGE_FEATURE_ENABLED",
environ_prefix=None,
)
FRONTEND_URL_JSON_FOOTER = values.Value(
None, environ_name="FRONTEND_URL_JSON_FOOTER", environ_prefix=None
)
FRONTEND_FOOTER_FEATURE_ENABLED = values.BooleanValue(
default=False,
environ_name="FRONTEND_FOOTER_FEATURE_ENABLED",
environ_prefix=None,
)
FRONTEND_FOOTER_VIEW_CACHE_TIMEOUT = values.Value(
60 * 60 * 24,
environ_name="FRONTEND_FOOTER_VIEW_CACHE_TIMEOUT",
environ_prefix=None,
)
FRONTEND_CSS_URL = values.Value(
None, environ_name="FRONTEND_CSS_URL", environ_prefix=None
)
# Posthog
POSTHOG_KEY = values.DictValue(
@@ -528,6 +549,9 @@ class Base(Configuration):
)
# AI service
AI_FEATURE_ENABLED = values.BooleanValue(
default=False, environ_name="AI_FEATURE_ENABLED", environ_prefix=None
)
AI_API_KEY = values.Value(None, environ_name="AI_API_KEY", environ_prefix=None)
AI_BASE_URL = values.Value(None, environ_name="AI_BASE_URL", environ_prefix=None)
AI_MODEL = values.Value(None, environ_name="AI_MODEL", environ_prefix=None)
@@ -586,14 +610,16 @@ class Base(Configuration):
LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"formatters": {
"simple": {
"format": "{asctime} {name} {levelname} {message}",
"style": "{",
},
},
"handlers": {
"console": {
"class": "logging.StreamHandler",
"level": values.Value(
"ERROR",
environ_name="LOGGING_LEVEL_HANDLERS_CONSOLE",
environ_prefix=None,
),
"formatter": "simple",
},
},
# Override root logger to send it to console

View File

@@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "impress"
version = "2.6.0"
version = "3.1.0"
authors = [{ "name" = "DINUM", "email" = "dev@mail.numerique.gouv.fr" }]
classifiers = [
"Development Status :: 5 - Production/Stable",
@@ -25,9 +25,10 @@ license = { file = "LICENSE" }
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
"boto3==1.37.5",
"beautifulsoup4==4.13.3",
"boto3==1.37.24",
"Brotli==1.1.0",
"celery[redis]==5.4.0",
"celery[redis]==5.5.0",
"django-configurations==2.5.1",
"django-cors-headers==4.7.0",
"django-countries==7.6.1",
@@ -37,26 +38,27 @@ dependencies = [
"django-redis==5.4.0",
"django-storages[s3]==1.14.5",
"django-timezone-field>=5.1",
"django==5.1.7",
"django==5.1.8",
"django-treebeard==4.7.1",
"djangorestframework==3.15.2",
"djangorestframework==3.16.0",
"drf_spectacular==0.28.0",
"dockerflow==2024.4.2",
"easy_thumbnails==2.10",
"factory_boy==3.3.3",
"gunicorn==23.0.0",
"jsonschema==4.23.0",
"lxml==5.3.1",
"markdown==3.7",
"mozilla-django-oidc==4.0.1",
"nested-multipart-parser==1.5.0",
"openai==1.65.2",
"psycopg[binary]==3.2.5",
"openai==1.70.0",
"psycopg[binary]==3.2.6",
"pycrdt==0.12.10",
"PyJWT==2.10.1",
"python-magic==0.4.27",
"requests==2.32.3",
"sentry-sdk==2.22.0",
"url-normalize==1.4.3",
"sentry-sdk==2.25.0",
"whitenoise==6.9.0",
"mozilla-django-oidc==4.0.1",
]
[project.urls]
@@ -72,18 +74,18 @@ dev = [
"drf-spectacular-sidecar==2025.3.1",
"freezegun==1.5.1",
"ipdb==0.13.13",
"ipython==9.0.1",
"pyfakefs==5.7.4",
"ipython==9.0.2",
"pyfakefs==5.8.0",
"pylint-django==2.6.1",
"pylint==3.3.4",
"pylint==3.3.6",
"pytest-cov==6.0.0",
"pytest-django==4.10.0",
"pytest==8.3.5",
"pytest-icdiff==0.9",
"pytest-xdist==3.6.1",
"responses==0.25.6",
"ruff==0.9.9",
"types-requests==2.32.0.20250301",
"responses==0.25.7",
"ruff==0.11.2",
"types-requests==2.32.0.20250328",
]
[tool.setuptools]

View File

@@ -1,5 +1,26 @@
import { Page, expect } from '@playwright/test';
export const CONFIG = {
AI_FEATURE_ENABLED: true,
CRISP_WEBSITE_ID: null,
COLLABORATION_WS_URL: 'ws://localhost:4444/collaboration/ws/',
ENVIRONMENT: 'development',
FRONTEND_CSS_URL: null,
FRONTEND_HOMEPAGE_FEATURE_ENABLED: true,
FRONTEND_FOOTER_FEATURE_ENABLED: true,
FRONTEND_THEME: 'default',
MEDIA_BASE_URL: 'http://localhost:8083',
LANGUAGES: [
['en-us', 'English'],
['fr-fr', 'Français'],
['de-de', 'Deutsch'],
['nl-nl', 'Nederlands'],
],
LANGUAGE_CODE: 'en-us',
POSTHOG_KEY: {},
SENTRY_DSN: null,
};
export const keyCloakSignIn = async (
page: Page,
browserName: string,

View File

@@ -2,24 +2,7 @@ import path from 'path';
import { expect, test } from '@playwright/test';
import { createDoc, verifyDocName } from './common';
const config = {
CRISP_WEBSITE_ID: null,
COLLABORATION_WS_URL: 'ws://localhost:8083/collaboration/ws/',
ENVIRONMENT: 'development',
FRONTEND_THEME: 'dsfr',
MEDIA_BASE_URL: 'http://localhost:8083',
LANGUAGES: [
['en-us', 'English'],
['fr-fr', 'Français'],
['de-de', 'Deutsch'],
['nl-nl', 'Nederlands'],
],
LANGUAGE_CODE: 'en-us',
POSTHOG_KEY: {},
SENTRY_DSN: null,
};
import { CONFIG, createDoc } from './common';
test.describe('Config', () => {
test('it checks the config api is called', async ({ page }) => {
@@ -33,7 +16,7 @@ test.describe('Config', () => {
const response = await responsePromise;
expect(response.ok()).toBeTruthy();
expect(await response.json()).toStrictEqual(config);
expect(await response.json()).toStrictEqual(CONFIG);
});
test('it checks that sentry is trying to init from config endpoint', async ({
@@ -44,7 +27,7 @@ test.describe('Config', () => {
if (request.method().includes('GET')) {
await route.fulfill({
json: {
...config,
...CONFIG,
SENTRY_DSN: 'https://sentry.io/123',
},
});
@@ -98,23 +81,46 @@ test.describe('Config', () => {
page,
browserName,
}) => {
const webSocketPromise = page.waitForEvent('websocket', (webSocket) => {
return webSocket.url().includes('ws://localhost:8083/collaboration/ws/');
await page.goto('/');
void createDoc(page, 'doc-collaboration', browserName, 1);
const webSocket = await page.waitForEvent('websocket', (webSocket) => {
return webSocket.url().includes('ws://localhost:4444/collaboration/ws/');
});
expect(webSocket.url()).toContain('ws://localhost:4444/collaboration/ws/');
});
test('it checks the AI feature flag from config endpoint', async ({
page,
browserName,
}) => {
await page.route('**/api/v1.0/config/', async (route) => {
const request = route.request();
if (request.method().includes('GET')) {
await route.fulfill({
json: {
...CONFIG,
AI_FEATURE_ENABLED: false,
},
});
} else {
await route.continue();
}
});
await page.goto('/');
const randomDoc = await createDoc(
page,
'doc-collaboration',
browserName,
1,
await createDoc(page, 'doc-ai-feature', browserName, 1);
await page.locator('.bn-block-outer').last().fill('Anything');
await page.getByText('Anything').selectText();
expect(
await page.locator('button[data-test="convertMarkdown"]').count(),
).toBe(1);
expect(await page.locator('button[data-test="ai-actions"]').count()).toBe(
0,
);
await verifyDocName(page, randomDoc[0]);
const webSocket = await webSocketPromise;
expect(webSocket.url()).toContain('ws://localhost:8083/collaboration/ws/');
});
test('it checks that Crisp is trying to init from config endpoint', async ({
@@ -125,7 +131,7 @@ test.describe('Config', () => {
if (request.method().includes('GET')) {
await route.fulfill({
json: {
...config,
...CONFIG,
CRISP_WEBSITE_ID: '1234',
},
});
@@ -140,6 +146,30 @@ test.describe('Config', () => {
page.locator('#crisp-chatbox').getByText('Invalid website'),
).toBeVisible();
});
test('it checks FRONTEND_CSS_URL config', async ({ page }) => {
await page.route('**/api/v1.0/config/', async (route) => {
const request = route.request();
if (request.method().includes('GET')) {
await route.fulfill({
json: {
...CONFIG,
FRONTEND_CSS_URL: 'http://localhost:123465/css/style.css',
},
});
} else {
await route.continue();
}
});
await page.goto('/');
await expect(
page
.locator('head link[href="http://localhost:123465/css/style.css"]')
.first(),
).toBeAttached();
});
});
test.describe('Config: Not loggued', () => {
@@ -159,7 +189,7 @@ test.describe('Config: Not loggued', () => {
expect(response.ok()).toBeTruthy();
const jsonResponse = await response.json();
expect(jsonResponse.FRONTEND_THEME).toStrictEqual('dsfr');
expect(jsonResponse.FRONTEND_THEME).toStrictEqual('default');
const footer = page.locator('footer').first();
// alt 'Gouvernement Logo' comes from the theme

View File

@@ -25,7 +25,11 @@ test.describe('Doc Editor', () => {
await editor.click();
await editor.fill('test content');
await editor.getByText('test content').dblclick();
await editor
.getByText('test content', {
exact: true,
})
.selectText();
const toolbar = page.locator('.bn-formatting-toolbar');
await expect(toolbar.locator('button[data-test="bold"]')).toBeVisible();
@@ -58,22 +62,22 @@ test.describe('Doc Editor', () => {
* - signal of the backend to the collaborative server (connection should close)
* - reconnection to the collaborative server
*/
test('checks the connection with collaborative server', async ({
page,
browserName,
}) => {
test('checks the connection with collaborative server', async ({ page }) => {
let webSocketPromise = page.waitForEvent('websocket', (webSocket) => {
return webSocket
.url()
.includes('ws://localhost:8083/collaboration/ws/?room=');
.includes('ws://localhost:4444/collaboration/ws/?room=');
});
const randomDoc = await createDoc(page, 'doc-editor', browserName, 1);
await verifyDocName(page, randomDoc[0]);
await page
.getByRole('button', {
name: 'New doc',
})
.click();
let webSocket = await webSocketPromise;
expect(webSocket.url()).toContain(
'ws://localhost:8083/collaboration/ws/?room=',
'ws://localhost:4444/collaboration/ws/?room=',
);
// Is connected
@@ -99,11 +103,11 @@ test.describe('Doc Editor', () => {
const wsClose = await wsClosePromise;
expect(wsClose.isClosed()).toBeTruthy();
// Checkt the ws is connected again
// Check the ws is connected again
webSocketPromise = page.waitForEvent('websocket', (webSocket) => {
return webSocket
.url()
.includes('ws://localhost:8083/collaboration/ws/?room=');
.includes('ws://localhost:4444/collaboration/ws/?room=');
});
webSocket = await webSocketPromise;
@@ -126,7 +130,7 @@ test.describe('Doc Editor', () => {
await expect(editor.getByText('[test markdown]')).toBeVisible();
await editor.getByText('[test markdown]').dblclick();
await editor.getByText('[test markdown]').selectText();
await page.locator('button[data-test="convertMarkdown"]').click();
await expect(editor.getByText('[test markdown]')).toBeHidden();
@@ -219,11 +223,8 @@ test.describe('Doc Editor', () => {
await editor.fill('Hello World Doc persisted 2');
await expect(editor.getByText('Hello World Doc persisted 2')).toBeVisible();
await page.goto('/');
await goToGridDoc(page, {
title: doc,
});
const urlDoc = page.url();
await page.goto(urlDoc);
await expect(editor.getByText('Hello World Doc persisted 2')).toBeVisible();
});
@@ -297,7 +298,7 @@ test.describe('Doc Editor', () => {
await page.locator('.bn-block-outer').last().fill('Hello World');
const editor = page.locator('.ProseMirror');
await editor.getByText('Hello').dblclick();
await editor.getByText('Hello').selectText();
await page.getByRole('button', { name: 'AI' }).click();
@@ -338,6 +339,7 @@ test.describe('Doc Editor', () => {
].forEach(({ ai_transform, ai_translate }) => {
test(`it checks AI buttons when can transform is at "${ai_transform}" and can translate is at "${ai_translate}"`, async ({
page,
browserName,
}) => {
await mockedDocument(page, {
accesses: [
@@ -364,16 +366,22 @@ test.describe('Doc Editor', () => {
link_reach: 'public',
link_role: 'editor',
created_at: '2021-09-01T09:00:00Z',
title: '',
});
await goToGridDoc(page);
const [randomDoc] = await createDoc(
page,
'doc-editor-ai',
browserName,
1,
);
await verifyDocName(page, 'Mocked document');
await verifyDocName(page, randomDoc);
await page.locator('.bn-block-outer').last().fill('Hello World');
const editor = page.locator('.ProseMirror');
await editor.getByText('Hello').dblclick();
await editor.getByText('Hello').selectText();
/* eslint-disable playwright/no-conditional-expect */
/* eslint-disable playwright/no-conditional-in-test */

View File

@@ -190,7 +190,7 @@ test.describe('Document grid item options', () => {
test.describe('Documents filters', () => {
test('it checks the prebuild left panel filters', async ({ page }) => {
await page.goto('/');
void page.goto('/');
// All Docs
const response = await page.waitForResponse(
@@ -263,7 +263,7 @@ test.describe('Documents filters', () => {
test.describe('Documents Grid', () => {
test('checks all the elements are visible', async ({ page }) => {
await page.goto('/');
void page.goto('/');
let docs: SmallDoc[] = [];
const response = await page.waitForResponse(

View File

@@ -86,7 +86,7 @@ test.describe('Document search', () => {
const editor = page.locator('.ProseMirror');
await editor.click();
await editor.fill('Hello world');
await editor.getByText('Hello world').dblclick();
await editor.getByText('Hello world').selectText();
await page.keyboard.press('Control+k');
await expect(page.getByRole('textbox', { name: 'Edit URL' })).toBeVisible();

View File

@@ -1,5 +1,7 @@
import { expect, test } from '@playwright/test';
import { CONFIG } from './common';
test.beforeEach(async ({ page }) => {
await page.goto('/docs/');
});
@@ -50,4 +52,27 @@ test.describe('Home page', () => {
await expect(footer).toBeVisible();
});
test('it checks the homepage feature flag', async ({ page }) => {
await page.route('**/api/v1.0/config/', async (route) => {
const request = route.request();
if (request.method().includes('GET')) {
await route.fulfill({
json: {
...CONFIG,
FRONTEND_HOMEPAGE_FEATURE_ENABLED: false,
},
});
} else {
await route.continue();
}
});
await page.goto('/');
// Keyclock login page
await expect(
page.locator('.login-pf-page-header').getByText('impress'),
).toBeVisible();
});
});

View File

@@ -1,6 +1,6 @@
{
"name": "app-e2e",
"version": "2.6.0",
"version": "3.1.0",
"private": true,
"scripts": {
"lint": "eslint . --ext .ts",

View File

@@ -1,492 +1,55 @@
const config = {
themes: {
default: {
theme: {
colors: {
'card-border': '#ededed',
'primary-bg': '#FAFAFA',
'primary-action': '#1212FF',
'primary-050': '#F5F5FE',
'primary-100': '#EDF5FA',
'primary-150': '#E5EEFA',
'primary-950': '#1B1B35',
'info-150': '#E5EEFA',
'greyscale-000': '#fff',
'greyscale-1000': '#161616',
'blue-400': '#7AB1E8',
'blue-500': '#417DC4',
'blue-600': '#3558A2',
'brown-400': '#E6BE92',
'brown-500': '#BD987A',
'brown-600': '#745B47',
'cyan-400': '#34BAB5',
'cyan-500': '#009099',
'cyan-600': '#006A6F',
'gold-400': '#FFCA00',
'gold-500': '#C3992A',
'gold-600': '#695240',
'green-400': '#34CB6A',
'green-500': '#00A95F',
'green-600': '#297254',
'olive-400': '#99C221',
'olive-500': '#68A532',
'olive-600': '#447049',
'orange-400': '#FF732C',
'orange-500': '#E4794A',
'orange-600': '#755348',
'pink-400': '#FFB7AE',
'pink-500': '#E18B76',
'pink-600': '#8D533E',
'purple-400': '#CE70CC',
'purple-500': '#A558A0',
'purple-600': '#6E445A',
'yellow-400': '#D8C634',
'yellow-500': '#B7A73F',
'yellow-600': '#66673D',
},
font: {
sizes: {
xs: '0.75rem',
sm: '0.875rem',
md: '1rem',
lg: '1.125rem',
ml: '0.938rem',
xl: '1.25rem',
t: '0.6875rem',
s: '0.75rem',
h1: '2rem',
h2: '1.75rem',
h3: '1.5rem',
h4: '1.375rem',
h5: '1.25rem',
h6: '1.125rem',
'xl-alt': '5rem',
'lg-alt': '4.5rem',
'md-alt': '4rem',
'sm-alt': '3.5rem',
'xs-alt': '3rem',
},
weights: {
thin: 100,
extrabold: 800,
black: 900,
},
},
spacings: {
'0': '0',
none: '0',
auto: 'auto',
bx: '2.2rem',
full: '100%',
'4xs': '0.125rem',
'3xs': '0.25rem',
'2xs': '0.375rem',
xs: '0.5rem',
sm: '0.75rem',
base: '1rem',
md: '1.5rem',
lg: '2rem',
xl: '2.5rem',
xxl: '3rem',
xxxl: '3.5rem',
'4xl': '4rem',
'5xl': '4.5rem',
'6xl': '6rem',
'7xl': '7.5rem',
},
breakpoints: {
xxs: '320px',
xs: '480px',
},
logo: {
src: '',
widthHeader: '',
widthFooter: '',
alt: '',
},
},
components: {
datagrid: {
header: {
weight: 'var(--c--theme--font--weights--extrabold)',
size: 'var(--c--theme--font--sizes--ml)',
},
cell: {
color: 'var(--c--theme--colors--primary-500)',
size: 'var(--c--theme--font--sizes--ml)',
},
},
'forms-checkbox': {
'background-color': {
hover: '#055fd214',
},
color: 'var(--c--theme--colors--primary-500)',
'font-size': 'var(--c--theme--font--sizes--ml)',
},
'forms-datepicker': {
'border-color': 'var(--c--theme--colors--primary-500)',
'value-color': 'var(--c--theme--colors--primary-500)',
'border-radius': {
hover: 'var(--c--components--forms-datepicker--border-radius)',
focus: 'var(--c--components--forms-datepicker--border-radius)',
},
},
'forms-field': {
color: 'var(--c--theme--colors--primary-500)',
'value-color': 'var(--c--theme--colors--primary-500)',
width: 'auto',
},
'forms-input': {
'value-color': 'var(--c--theme--colors--primary-500)',
'border-color': 'var(--c--theme--colors--primary-500)',
color: {
error: 'var(--c--theme--colors--danger-500)',
'error-hover': 'var(--c--theme--colors--danger-500)',
'box-shadow-error-hover': 'var(--c--theme--colors--danger-500)',
},
},
'forms-labelledbox': {
'label-color': {
small: 'var(--c--theme--colors--primary-500)',
'small-disabled': 'var(--c--theme--colors--greyscale-400)',
big: {
disabled: 'var(--c--theme--colors--greyscale-400)',
},
},
},
'forms-select': {
'border-color': 'var(--c--theme--colors--primary-500)',
'border-color-disabled-hover':
'var(--c--theme--colors--greyscale-200)',
'border-radius': {
hover: 'var(--c--components--forms-select--border-radius)',
focus: 'var(--c--components--forms-select--border-radius)',
},
'font-size': 'var(--c--theme--font--sizes--ml)',
'menu-background-color': '#fff',
'item-background-color': {
hover: 'var(--c--theme--colors--primary-300)',
},
},
'forms-switch': {
'accent-color': 'var(--c--theme--colors--primary-400)',
},
'forms-textarea': {
'border-color': 'var(--c--components--forms-textarea--border-color)',
'border-color-hover':
'var(--c--components--forms-textarea--border-color)',
'border-radius': {
hover: 'var(--c--components--forms-textarea--border-radius)',
focus: 'var(--c--components--forms-textarea--border-radius)',
},
color: 'var(--c--theme--colors--primary-500)',
disabled: {
'border-color-hover': 'var(--c--theme--colors--greyscale-200)',
},
},
modal: {
'background-color': '#fff',
},
button: {
'border-radius': {
active: 'var(--c--components--button--border-radius)',
},
'medium-height': 'auto',
'small-height': 'auto',
success: {
color: 'white',
'color-disabled': 'white',
'color-hover': 'white',
background: {
color: 'var(--c--theme--colors--success-600)',
'color-disabled': 'var(--c--theme--colors--greyscale-300)',
'color-hover': 'var(--c--theme--colors--success-800)',
},
},
danger: {
'color-hover': 'white',
background: {
color: 'var(--c--theme--colors--danger-600)',
'color-hover': '#FF2725',
'color-disabled': 'var(--c--theme--colors--danger-100)',
},
},
primary: {
color: 'var(--c--theme--colors--primary-text)',
'color-active': 'var(--c--theme--colors--primary-text)',
background: {
color: 'var(--c--theme--colors--primary-400)',
'color-active': 'var(--c--theme--colors--primary-500)',
},
border: {
'color-active': 'transparent',
},
},
secondary: {
color: 'var(--c--theme--colors--primary-500)',
'color-hover': 'var(--c--theme--colors--primary-text)',
background: {
color: 'white',
'color-hover': 'var(--c--theme--colors--primary-700)',
},
border: {
color: 'var(--c--theme--colors--greyscale-300)',
},
},
tertiary: {
color: 'var(--c--theme--colors--primary-text)',
'color-disabled': 'var(--c--theme--colors--greyscale-600)',
background: {
color: 'var(--c--theme--colors--primary-100)',
'color-hover': 'var(--c--theme--colors--primary-300)',
'color-active': 'var(--c--theme--colors--primary-100)',
'color-disabled': 'var(--c--theme--colors--greyscale-200)',
},
},
disabled: {
color: 'white',
background: {
color: '#b3cef0',
},
},
},
'la-gauffre': {
activated: false,
},
'home-proconnect': {
activated: false,
},
},
import { cunninghamConfig as tokens } from '@gouvfr-lasuite/ui-kit';
const customColors = {
'primary-action': '#1212FF',
'primary-bg': '#FAFAFA',
'blue-400': '#7AB1E8',
'blue-500': '#417DC4',
'blue-600': '#3558A2',
'brown-400': '#E6BE92',
'brown-500': '#BD987A',
'brown-600': '#745B47',
'cyan-400': '#34BAB5',
'cyan-500': '#009099',
'cyan-600': '#006A6F',
'gold-400': '#FFCA00',
'gold-500': '#C3992A',
'gold-600': '#695240',
'green-400': '#34CB6A',
'green-500': '#00A95F',
'green-600': '#297254',
'olive-400': '#99C221',
'olive-500': '#68A532',
'olive-600': '#447049',
'orange-400': '#FF732C',
'orange-500': '#E4794A',
'orange-600': '#755348',
'pink-400': '#FFB7AE',
'pink-500': '#E18B76',
'pink-600': '#8D533E',
'purple-400': '#CE70CC',
'purple-500': '#A558A0',
'purple-600': '#6E445A',
'yellow-400': '#D8C634',
'yellow-500': '#B7A73F',
'yellow-600': '#66673D',
};
tokens.themes.default.theme.colors = {
...tokens.themes.default.theme.colors,
...customColors,
};
tokens.themes.default.components = {
...tokens.themes.default.components,
...{
'la-gauffre': {
activated: true,
},
dsfr: {
theme: {
colors: {
'card-border': '#E5E5E5',
'primary-text': '#000091',
'primary-100': '#ECECFE',
'primary-150': '#F4F4FD',
'primary-200': '#E3E3FD',
'primary-300': '#CACAFB',
'primary-400': '#8585F6',
'primary-500': '#6A6AF4',
'primary-600': '#313178',
'primary-700': '#272747',
'primary-800': '#000091',
'primary-900': '#21213F',
'secondary-text': '#fff',
'secondary-100': '#fee9ea',
'secondary-200': '#fedfdf',
'secondary-300': '#fdbfbf',
'secondary-400': '#e1020f',
'secondary-500': '#c91a1f',
'secondary-600': '#5e2b2b',
'secondary-700': '#3b2424',
'secondary-800': '#341f1f',
'secondary-900': '#2b1919',
'greyscale-text': '#303C4B',
'greyscale-000': '#fff',
'greyscale-050': '#F6F6F6',
'greyscale-100': '#eee',
'greyscale-200': '#E5E5E5',
'greyscale-250': '#ddd',
'greyscale-300': '#CECECE',
'greyscale-350': '#ddd',
'greyscale-400': '#929292',
'greyscale-500': '#7C7C7C',
'greyscale-600': '#666666',
'greyscale-700': '#3A3A3A',
'greyscale-750': '#353535',
'greyscale-800': '#2A2A2A',
'greyscale-900': '#242424',
'greyscale-950': '#1E1E1E',
'greyscale-1000': '#161616',
'success-text': '#1f8d49',
'success-100': '#dffee6',
'success-200': '#b8fec9',
'success-300': '#88fdaa',
'success-400': '#3bea7e',
'success-500': '#1f8d49',
'success-600': '#18753c',
'success-700': '#204129',
'success-800': '#1e2e22',
'success-900': '#19281d',
'info-text': '#0078f3',
'info-100': '#E8EDFF',
'info-200': '#DDE5FF',
'info-300': '#BCCDFF',
'info-400': '#518FFF',
'info-500': '#0078F3',
'info-600': '#0063CB',
'info-700': '#273961',
'info-800': '#222A3F',
'info-900': '#1D2437',
'warning-text': '#d64d00',
'warning-100': '#fff4f3',
'warning-200': '#ffe9e6',
'warning-300': '#ffded9',
'warning-400': '#ffbeb4',
'warning-500': '#d64d00',
'warning-600': '#b34000',
'warning-700': '#5e2c21',
'warning-800': '#3e241e',
'warning-900': '#361e19',
'danger-text': '#FFF',
'danger-100': '#FFE9E9',
'danger-200': '#FFDDDD',
'danger-300': '#FFBDBD',
'danger-400': '#FF5655',
'danger-500': '#F60700',
'danger-600': '#CE0500',
'danger-700': '#642626',
'danger-800': '#412121',
'danger-900': '#391C1C',
},
font: {
families: {
accent: 'Marianne',
base: 'Marianne',
},
},
logo: {
src: '/assets/logo-gouv.svg',
widthHeader: '110px',
widthFooter: '220px',
alt: 'Gouvernement Logo',
},
},
components: {
alert: {
'border-radius': '0',
error: {
'background-color': 'var(--c--theme--colors--danger-100)',
'border-left-color': 'var(--c--theme--colors--danger-400)',
close: {
color: 'white',
'background-color': 'var(--c--theme--colors--danger-400)',
'background-color-hover': 'var(--c--theme--colors--danger-600)',
},
},
},
modal: {
'width-small': '342px',
},
button: {
'medium-height': '40px',
'medium-text-height': '40px',
'border-radius': '4px',
primary: {
background: {
color: 'var(--c--theme--colors--primary-text)',
'color-hover': '#1212ff',
'color-active': '#2323ff',
},
color: '#fff',
'color-hover': '#fff',
'color-active': '#fff',
},
'primary-text': {
background: {
'color-hover': 'var(--c--theme--colors--primary-100)',
'color-active': 'var(--c--theme--colors--primary-100)',
},
'color-hover': 'var(--c--theme--colors--primary-text)',
color: 'var(--c--theme--colors--primary-800)',
},
secondary: {
background: {
'color-hover': '#F6F6F6',
'color-active': '#EDEDED',
},
border: {
color: 'var(--c--theme--colors--greyscale-300)',
'color-hover': 'var(--c--theme--colors--greyscale-300)',
},
color: 'var(--c--theme--colors--primary-text)',
},
'tertiary-text': {
background: {
'color-hover': 'var(--c--theme--colors--greyscale-100)',
},
'color-hover': 'var(--c--theme--colors--primary-text)',
color: 'var(--c--theme--colors--primary-600)',
},
},
datagrid: {
header: {
color: 'var(--c--theme--colors--primary-text)',
size: 'var(--c--theme--font--sizes--s)',
},
body: {
'background-color': 'transparent',
'background-color-hover': '#F4F4FD',
},
pagination: {
'background-color': 'transparent',
'background-color-active': 'var(--c--theme--colors--primary-300)',
'border-color': 'var(--c--theme--colors--primary-400)',
},
},
'forms-checkbox': {
'border-radius': '0',
color: 'var(--c--theme--colors--primary-text)',
text: {
color: 'var(--c--theme--colors--greyscale-text)',
size: 'var(--c--theme--font--sizes--t)',
},
},
'forms-datepicker': {
'border-radius': '0',
},
'forms-fileuploader': {
'border-radius': '0',
},
'forms-field': {
color: 'var(--c--theme--colors--primary-text)',
'footer-font-size': 'var(--c--theme--font--sizes--t)',
'footer-color': 'var(--c--theme--colors--greyscale-text)',
},
'forms-input': {
'border-radius': '4px',
'background-color': '#fff',
'border-color': 'var(--c--theme--colors--primary-text)',
'box-shadow-color': 'var(--c--theme--colors--primary-text)',
'value-color': 'var(--c--theme--colors--primary-text)',
'font-size': '14px',
},
'forms-labelledbox': {
'label-color': {
big: 'var(--c--theme--colors--primary-text)',
},
},
'forms-radio': {
'accent-color': 'var(--c--theme--colors--primary-600)',
},
'forms-select': {
'item-font-size': '14px',
'border-radius': '4px',
'border-radius-hover': '4px',
'background-color': '#fff',
'border-color': 'var(--c--theme--colors--primary-text)',
'border-color-hover': 'var(--c--theme--colors--primary-text)',
'box-shadow-color': 'var(--c--theme--colors--primary-text)',
},
'forms-switch': {
'handle-border-radius': '2px',
'rail-border-radius': '4px',
'accent-color': 'var(--c--theme--colors--primary-text)',
},
'forms-textarea': {
'border-radius': '0',
},
'la-gauffre': {
activated: true,
},
'home-proconnect': {
activated: true,
},
},
'home-proconnect': {
activated: true,
},
},
};
export default config;
export default tokens;

View File

@@ -9,6 +9,7 @@ const createJestConfig = nextJest({
const config: Config = {
coverageProvider: 'v8',
moduleNameMapper: {
'^@/docs/(.*)$': '<rootDir>/src/features/docs/$1',
'^@/(.*)$': '<rootDir>/src/$1',
},
setupFilesAfterEnv: ['<rootDir>/jest.setup.ts'],

View File

@@ -1,6 +1,6 @@
{
"name": "app-impress",
"version": "2.6.0",
"version": "3.1.0",
"private": true,
"scripts": {
"dev": "next dev",
@@ -21,13 +21,16 @@
"@blocknote/react": "0.23.2-hotfix.0",
"@blocknote/xl-docx-exporter": "0.23.2-hotfix.0",
"@blocknote/xl-pdf-exporter": "0.23.2-hotfix.0",
"@fontsource/material-icons": "5.2.5",
"@gouvfr-lasuite/integration": "1.0.2",
"@gouvfr-lasuite/ui-kit": "0.1.3",
"@hocuspocus/provider": "2.15.2",
"@openfun/cunningham-react": "3.0.0",
"@react-pdf/renderer": "4.1.6",
"@sentry/nextjs": "9.3.0",
"@tanstack/react-query": "5.67.1",
"canvg": "4.0.3",
"clsx": "2.1.1",
"cmdk": "1.0.4",
"crisp-sdk-web": "1.0.25",
"docx": "9.1.1",
@@ -36,7 +39,7 @@
"idb": "8.0.2",
"lodash": "4.17.21",
"luxon": "3.5.0",
"next": "15.2.1",
"next": "15.2.4",
"posthog-js": "1.227.0",
"react": "*",
"react-aria-components": "1.6.0",

View File

@@ -0,0 +1,121 @@
{
"default": {
"externalLinks": [
{
"label": "Github",
"href": "https://github.com/suitenumerique/docs/"
},
{
"label": "DINUM",
"href": "https://www.numerique.gouv.fr/dinum/"
},
{
"label": "ZenDiS",
"href": "https://zendis.de/"
},
{
"label": "BlockNote.js",
"href": "https://www.blocknotejs.org/"
}
],
"bottomInformation": {
"label": "Unless otherwise stated, all content on this site is under",
"link": {
"label": "licence etalab-2.0",
"href": "https://github.com/etalab/licence-ouverte/blob/master/LO.md"
}
}
},
"en": {
"legalLinks": [
{
"label": "Legal Notice",
"href": "#"
},
{
"label": "Personal data and cookies",
"href": "#"
},
{
"label": "Accessibility",
"href": "#"
}
],
"bottomInformation": {
"label": "Unless otherwise stated, all content on this site is under",
"link": {
"label": "licence MIT",
"href": "https://github.com/suitenumerique/docs/blob/main/LICENSE"
}
}
},
"fr": {
"legalLinks": [
{
"label": "Mentions légales",
"href": "#"
},
{
"label": "Données personnelles et cookies",
"href": "#"
},
{
"label": "Accessibilité",
"href": "#"
}
],
"bottomInformation": {
"label": "Sauf mention contraire, tout le contenu de ce site est sous",
"link": {
"label": "licence MIT",
"href": "https://github.com/suitenumerique/docs/blob/main/LICENSE"
}
}
},
"de": {
"legalLinks": [
{
"label": "Impressum",
"href": "#"
},
{
"label": "Personenbezogene Daten und Cookies",
"href": "#"
},
{
"label": "Barrierefreiheit",
"href": "#"
}
],
"bottomInformation": {
"label": "Sofern nicht anders angegeben, steht der gesamte Inhalt dieser Website unter",
"link": {
"label": "licence MIT",
"href": "https://github.com/suitenumerique/docs/blob/main/LICENSE"
}
}
},
"nl": {
"legalLinks": [
{
"label": "Wettelijke bepalingen",
"href": "#"
},
{
"label": "Persoonlijke gegevens en cookies",
"href": "#"
},
{
"label": "Toegankelijkheid",
"href": "#"
}
],
"bottomInformation": {
"label": "Tenzij anders vermeld, is alle inhoud van deze site ondergebracht onder",
"link": {
"label": "licence MIT",
"href": "https://github.com/suitenumerique/docs/blob/main/LICENSE"
}
}
}
}

View File

@@ -0,0 +1,135 @@
{
"default": {
"externalLinks": [
{
"label": "legifrance.gouv.fr",
"href": "https://legifrance.gouv.fr/"
},
{
"label": "info.gouv.fr",
"href": "https://info.gouv.fr/"
},
{
"label": "service-public.fr",
"href": "https://service-public.fr/"
},
{
"label": "data.gouv.fr",
"href": "https://data.gouv.fr/"
}
],
"legalLinks": [
{
"label": "Legal Notice",
"href": "https://docs.numerique.gouv.fr/docs/4db744e3-5b47-4ed9-b9e7-d4312318bbce/"
},
{
"label": "Personal data and cookies",
"href": "https://docs.numerique.gouv.fr/docs/eb863389-a5e5-4d18-879d-149a1122380e/"
},
{
"label": "Accessibility",
"href": "https://docs.numerique.gouv.fr/docs/9694e570-1427-4ef7-b0a0-c3e894360e1b/"
}
],
"bottomInformation": {
"label": "Unless otherwise stated, all content on this site is under",
"link": {
"label": "licence etalab-2.0",
"href": "https://github.com/etalab/licence-ouverte/blob/master/LO.md"
}
}
},
"en": {
"legalLinks": [
{
"label": "Legal Notice",
"href": "https://docs.numerique.gouv.fr/docs/4db744e3-5b47-4ed9-b9e7-d4312318bbce/"
},
{
"label": "Personal data and cookies",
"href": "https://docs.numerique.gouv.fr/docs/eb863389-a5e5-4d18-879d-149a1122380e/"
},
{
"label": "Accessibility",
"href": "https://docs.numerique.gouv.fr/docs/9694e570-1427-4ef7-b0a0-c3e894360e1b/"
}
],
"bottomInformation": {
"label": "Unless otherwise stated, all content on this site is under",
"link": {
"label": "licence etalab-2.0",
"href": "https://github.com/etalab/licence-ouverte/blob/master/LO.md"
}
}
},
"fr": {
"legalLinks": [
{
"label": "Mentions légales",
"href": "https://docs.numerique.gouv.fr/docs/4db744e3-5b47-4ed9-b9e7-d4312318bbce/"
},
{
"label": "Données personnelles et cookies",
"href": "https://docs.numerique.gouv.fr/docs/eb863389-a5e5-4d18-879d-149a1122380e/"
},
{
"label": "Accessibilité",
"href": "https://docs.numerique.gouv.fr/docs/9694e570-1427-4ef7-b0a0-c3e894360e1b/"
}
],
"bottomInformation": {
"label": "Sauf mention contraire, tout le contenu de ce site est sous",
"link": {
"label": "licence etalab-2.0",
"href": "https://github.com/etalab/licence-ouverte/blob/master/LO.md"
}
}
},
"de": {
"legalLinks": [
{
"label": "Impressum",
"href": "https://docs.numerique.gouv.fr/docs/4db744e3-5b47-4ed9-b9e7-d4312318bbce/"
},
{
"label": "Personenbezogene Daten und Cookies",
"href": "https://docs.numerique.gouv.fr/docs/eb863389-a5e5-4d18-879d-149a1122380e/"
},
{
"label": "Barrierefreiheit",
"href": "https://docs.numerique.gouv.fr/docs/9694e570-1427-4ef7-b0a0-c3e894360e1b/"
}
],
"bottomInformation": {
"label": "Sofern nicht anders angegeben, steht der gesamte Inhalt dieser Website unter",
"link": {
"label": "licence etalab-2.0",
"href": "https://github.com/etalab/licence-ouverte/blob/master/LO.md"
}
}
},
"nl": {
"legalLinks": [
{
"label": "Wettelijke bepalingen",
"href": "https://docs.numerique.gouv.fr/docs/4db744e3-5b47-4ed9-b9e7-d4312318bbce/"
},
{
"label": "Persoonlijke gegevens en cookies",
"href": "https://docs.numerique.gouv.fr/docs/eb863389-a5e5-4d18-879d-149a1122380e/"
},
{
"label": "Toegankelijkheid",
"href": "https://docs.numerique.gouv.fr/docs/9694e570-1427-4ef7-b0a0-c3e894360e1b/"
}
],
"bottomInformation": {
"label": "Tenzij anders vermeld, is alle inhoud van deze site ondergebracht onder",
"link": {
"label": "licence etalab-2.0",
"href": "https://github.com/etalab/licence-ouverte/blob/master/LO.md"
}
}
}
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 992 B

View File

@@ -1,101 +0,0 @@
@font-face {
font-family: Marianne;
src:
url('Marianne-Thin.woff2') format('woff2'),
url('Marianne-Thin.woff') format('woff');
font-weight: 100;
}
@font-face {
font-family: Marianne;
src:
url('Marianne-Thin_Italic.woff2') format('woff2'),
url('Marianne-Thin_Italic.woff') format('woff');
font-weight: 100;
font-style: italic;
}
@font-face {
font-family: Marianne;
src:
url('Marianne-Light.woff2') format('woff2'),
url('Marianne-Light.woff') format('woff');
font-weight: 300;
}
@font-face {
font-family: Marianne;
src:
url('Marianne-Light_Italic.woff2') format('woff2'),
url('Marianne-Light_Italic.woff') format('woff');
font-weight: 300;
font-style: italic;
}
@font-face {
font-family: Marianne;
src:
url('Marianne-Regular.woff2') format('woff2'),
url('Marianne-Regular.woff') format('woff');
font-weight: 400;
}
@font-face {
font-family: Marianne;
src:
url('Marianne-Regular_Italic.woff2') format('woff2'),
url('Marianne-Regular_Italic.woff') format('woff');
font-weight: 400;
font-style: italic;
}
@font-face {
font-family: Marianne;
src:
url('Marianne-Medium.woff2') format('woff2'),
url('Marianne-Medium.woff') format('woff');
font-weight: 500;
}
@font-face {
font-family: Marianne;
src:
url('Marianne-Medium_Italic.woff2') format('woff2'),
url('Marianne-Medium_Italic.woff') format('woff');
font-weight: 500;
font-style: italic;
}
@font-face {
font-family: Marianne;
src:
url('Marianne-Bold.woff2') format('woff2'),
url('Marianne-Bold.woff') format('woff');
font-weight: 700;
}
@font-face {
font-family: Marianne;
src:
url('Marianne-Bold_Italic.woff2') format('woff2'),
url('Marianne-Bold_Italic.woff') format('woff');
font-weight: 700;
font-style: italic;
}
@font-face {
font-family: Marianne;
src:
url('Marianne-ExtraBold.woff2') format('woff2'),
url('Marianne-ExtraBold.woff') format('woff');
font-weight: 800;
}
@font-face {
font-family: Marianne;
src:
url('Marianne-ExtraBold_Italic.woff2') format('woff2'),
url('Marianne-ExtraBold_Italic.woff') format('woff');
font-weight: 800;
font-style: italic;
}

Some files were not shown because too many files have changed in this diff Show More