Compare commits

...

231 Commits

Author SHA1 Message Date
Nathan Panchout
2b2a579d9f wip 2025-03-24 17:15:14 +01:00
Nathan Panchout
615ab564cc fixup! (frontend) refactor and theme token update 2025-03-24 09:00:59 +01:00
Nathan Panchout
aadd6d9ec3 fixup! (frontend) added subpage management and document tree features 2025-03-24 08:55:38 +01:00
Nathan Panchout
0a6502a77d (frontend) added subpage management and document tree features
New components were created to manage subpages in the document tree,
including the ability to add, reorder, and view subpages. Tests were
added to verify the functionality of these features. Additionally, API
changes were made to manage the creation and retrieval of document
children.
2025-03-24 08:55:36 +01:00
Nathan Panchout
a32ee20249 (Frontend) Added drag-and-drop functionality for document management
Added a new feature for moving documents within the user interface via
drag-and-drop. This includes the creation of Draggable and Droppable
components, as well as tests to verify document creation and movement
behavior. Changes have also been made to document types to include user
roles and child management capabilities.
2025-03-24 08:54:03 +01:00
Nathan Panchout
48db42f385 (frontend) refactor and theme token update
The configuration file has been simplified by importing configurations
from @gouvfr-lasuite/ui-kit . Colors and components have been updated to
reflect the new values. Additionally, adjustments have been made to
global styles, including the addition of styles for Material icons. Form
components have also been modified to incorporate the new style
properties.
2025-03-24 08:54:01 +01:00
Nathan Panchout
05b14b2948 (frontend) updated dependencies and added new packages
Added several new dependencies to the `package.json` file, including
`@dnd-kit/core`, `@dnd-kit/modifiers`, `@fontsource/material-icons`, and
`@gouvfr-lasuite/ui-kit`.
2025-03-24 08:53:04 +01:00
Nathan Panchout
12f4a72f5e 🐛(back) keep info if document has deleted children
With the soft delete feature, relying on the is_leaf method from the
treebeard is not accurate anymore. To determine if a node is a leaf, it
checks if the number of numchild is equal to 0. But a node can have soft
deleted children, then numchild is equal to 0, but it is not a leaf
because if we want to add a child we have to look for the last child to
compute a correct path. Otherwise we will have an error saying that the
path already exists.
2025-03-24 08:51:42 +01:00
Bastien Guerry
ef7cc67387 📄(legal) Require contributors to sign a DCO
Contributors are required to sign off their commits: this confirms
that they have read and accepted https://developercertificate.org.
2025-03-23 09:57:35 +01:00
Sylvain Zimmer
a8529e434a 🐛(media) fix compatibility with Scaleway Object Storage
Some providers with S3-compatible APIs have slightly different
implementations. In this case, Scaleway didn't accept version_id=""
and has a different version ID scheme. This was tested successfully
and should remain compatible with any other provider.
2025-03-22 18:00:43 +01:00
Manuel Raynaud
f8203a1766 🚨(back) lint code with ruff 0.11.2
New Ruff rule (C420) detects code that should be linted. We apply this
new rule on our code.
2025-03-22 10:28:48 +01:00
renovate[bot]
ce8b98e256 ⬆️(dependencies) update python dependencies 2025-03-22 10:28:48 +01:00
Anthony LC
4243519eee 🔥(frontend) remove Marianne font
Marianne font is now part of the UI kit.
We can remove it from the project.
2025-03-21 17:49:06 +01:00
Nathan Panchout
1abf529891 (frontend) refactor and theme token update
The configuration file has been simplified by importing configurations
from @gouvfr-lasuite/ui-kit . Colors and components have been updated to
reflect the new values. Additionally, adjustments have been made to
global styles, including the addition of styles for Material icons. Form
components have also been modified to incorporate the new style
properties.
2025-03-21 17:49:06 +01:00
Nathan Panchout
69ca4af539 (frontend) updated dependencies and added new packages
Added several new dependencies to the `package.json` file, including
`@dnd-kit/core`, `@dnd-kit/modifiers`, `@fontsource/material-icons`, and
`@gouvfr-lasuite/ui-kit`.
2025-03-21 17:49:06 +01:00
Anthony LC
14b2adedfb 🔖(minor) release 2.6.0
Added:
- 📝(doc) add publiccode.yml

Changed
- 🚸(frontend) ctrl+k modal not when editor is focused

Fixed:
- 🐛(back) allow only images to be used with
  the cors-proxy
- 🐛(backend) stop returning inactive users
  on the list endpoint
- 🔒️(backend) require at least 5 characters
  to search for users
- 🔒️(back) throttle user list endpoint
- 🔒️(back) remove pagination and limit to
   5 for user list endpoint
2025-03-21 17:07:26 +01:00
Anthony LC
a7edb382a7 🩹(frontent) change selector to block cmd+k
Multiple ctrl+k could open the search modal, we
change the selector, now if the toolbar is displayed
we don't open the search modal.
2025-03-21 17:07:26 +01:00
Anthony LC
fb5400c26b ️(frontend) search users with at least 5 characters
We now only search for users when the query
is at least 5 characters long.
2025-03-21 15:44:09 +01:00
Manuel Raynaud
8473facbee 🔒️(back) throttle user list endpoint
The user list endpoint is throttle to avoid users discovery. The
throttle is set to 500 requests per day. This can be changed using the
settings API_USERS_LIST_THROTTLE_RATE.
2025-03-21 15:44:09 +01:00
Anthony LC
5db446e8a8 🏷️(frontend) adapt type for user search
The response from the user request is now an
array of users, we don't paginate anymore.
We adapt the types to reflect this.
2025-03-21 15:44:09 +01:00
Manuel Raynaud
34dfb3fd66 🔒️(back) remove pagination and limit to 5 for user list endpoint
The user list endpoint does not use anymore a pagination, the results is
directly return in a list and the max results returned is limited to 5.
In order to modify this limit the settings API_USERS_LIST_LIMIT is
used.
2025-03-21 15:44:09 +01:00
Samuel Paccoud - DINUM
f9a91eda2d 🐛(backend) stop returning inactive users on the list endpoint
inactive users should not be returned as we don't want users to be
able to share new documents with them.
2025-03-21 15:44:09 +01:00
Samuel Paccoud - DINUM
eba926dea4 🔒️(backend) require at least 5 characters to search for users
Listing users is made a little to easy for authenticated users.
2025-03-21 15:44:09 +01:00
Anthony LC
3839a2e8b1 💄(frontend) improve contrast of Beta icon
The colors of the Beta icon were not contrasted
enough. This was posing an accessibility issue.
We now use a more contrasted color.
2025-03-21 09:22:42 +01:00
Anthony LC
a88d62e07d 🌐(frontend) make Docs title translatable
The title of the docs page was not translatable.
We now use the `t` function to translate the title.
2025-03-21 09:22:42 +01:00
Paul Mustière
b61a7a4961 📝(docs) fix typo
Correct language to not be past tense
2025-03-21 06:38:27 +01:00
Anthony LC
20d32ecc4e 🚸(frontend) ctrl+k modal not when editor is focused
ctrl+k interaction was as well used in the editor.
So if the user has a focus on the editor, we don't
open the searchmodal.
2025-03-20 17:43:32 +01:00
Manuel Raynaud
313acf4f78 🐛(back) allow only images to be used with the cors-proxy
The cors-proxy endpoint allowed to use every type of files and to
execute it in the browser. We limit the scope only to images and
Content-Security-Policy and Content-Disposition headers are also added
to not allow script execution that can be present in a SVG file.
2025-03-20 16:10:47 +01:00
Bastien
3a6105cc7e 📝(doc) add publiccode.yml (#770)
publiccode.yml is a standard for describing Free Software projects,
similar to other initiatives such as https://codemeta.github.io.

It is particularly suitable for describing projects funded by public
administrations. See https://github.com/publiccodeyml/publiccode.yml
2025-03-19 21:28:32 +01:00
Anthony LC
bbe17156be 🔖(minor) release 2.5.0
Added:
- 📝(doc) Added GNU Make link to README
- (frontend) add pinning on doc detail
- 🚩(frontend) feature flag analytic on copy as html
- (frontend) Custom block divider with export
- 🌐(i18n) activate dutch language

Changed:
- 🧑‍💻(frontend) change literal section open source
- ♻️(frontend) replace cors proxy for export
- 🚨(gitlint) Allow uppercase in commit messages

Fixed:
- 🐛(frontend) SVG export
- 🐛(frontend) remove scroll listener table content
- 🔒️(back) restrict access to favorite_list endpoint
- 🐛(backend) refactor to fix filtering on children
    and descendants views
- 🐛(action) fix notify-argocd workflow
- 🚨(helm) fix helmfile lint
- 🚚(frontend) redirect to 401 page when 401 error
2025-03-19 14:11:47 +01:00
Anthony LC
51cc26b916 🐛(frontend) improve svg export to be less pixelized
Some SVGs were pixelized in the exported files.
We now add the wanted size to the svg conversion to
make sure the images are exported with the correct size
and so less pixelized.
2025-03-19 14:11:47 +01:00
Anthony LC
cab8ef51df 🐛(frontend) unmount components Analytics
`useAnalytics` hooks was dispatching methods that
caused children components to be unmounted.
By declaring the methods out of the hook, we can
prevent the components from being unmounted.
2025-03-18 14:53:09 +01:00
Anthony LC
6627518017 🚚(frontend) redirect to 401 page when 401 error
Users could still be able to edit a document if the
session was expired. It could give the feeling that the
document was not saved.
If during a mutation request (POST, PUT, DELETE),
the server returns a 401 error,
the user is redirected to the 401 page.
2025-03-18 14:53:09 +01:00
Pedro Manse
12c18bc4e9 📝(README) Add link to GNU Make
Just like docker-compose, create link to the software's site on it's
first mention.

📝(Changelog) Added entry

📝(Changelog) Added pull request id
2025-03-18 11:07:22 +01:00
Anthony LC
aff330eb5b 🚨(gitlint) Allow uppercase in commit messages
Many developers use uppercase as the first letter
in their commit messages, it creates an error.
We will allow uppercase in commit messages to
lower frustration when committing.
2025-03-18 10:24:08 +01:00
Cameron King
bcdaedba9b 🐛(backend) add user/db to pg healthchecks
Adds PostgreSQL user and database names to the docker-compose.yaml healthchecks.
This resolves an error that appears in the logs, where 'root' is used by
default.
2025-03-18 09:41:27 +01:00
Manuel Raynaud
799814e3e3 🌐(i18n) activate dutch language
All the dutch translations are complete on crowdin. We activate it in
the django settings and download all translations from crowdin
2025-03-18 09:27:13 +01:00
virgile-dev
02c9b2ea2e 🐛(readme) fix preprod link to redirect to homepage (#747)
The current link redirects to a 404. New link redirect to homepage.
2025-03-17 16:02:45 +00:00
Manuel Raynaud
eb23aefd55 ♻️(back) use same base route path for swagger
Swaggers urls where not using the same base route path /api/v1.0, we
prepend it to have the same path everywhere. Moreover, a double slash
was used for swagger and redoc dashboard.
2025-03-17 15:02:34 +01:00
Manuel Raynaud
0c49019490 🚨(helm) fix helmfile lint
Latest release of helmfile is applying the change related before as a
warning. Environnements must be before releases but not in the same
document of repositories.
2025-03-17 14:40:55 +01:00
Anthony LC
170dbe07bb ⬆️(frontend) bump @babel/runtime /src/frontend
Bumps @babel/runtime from 7.26.7 to 7.26.10.
2025-03-17 13:50:20 +01:00
Manuel Raynaud
70136f2415 🐛(action) fix notify-argocd workflow
The notify-argocd workflow was not working correctly. The html_url sent
to argocd was not the good one anymore.
2025-03-17 12:09:18 +01:00
Anthony LC
2a8fc97f2f ⬆️(frontend) bump @babel/helpers in /src/frontend
Bumps @babel/helpers from 7.26.7 to 7.26.10.
2025-03-17 11:50:22 +01:00
Anthony LC
9570701bc3 ⬆️(frontend) bump @babel/runtime /src/mail
Bumps @babel/runtime from 7.26.0 to 7.26.10.
2025-03-17 11:36:04 +01:00
Anthony LC
4b28b3c23b (frontend) add pinning on doc detail
Add pinning button on doc detail page.
2025-03-17 11:16:50 +01:00
Anthony LC
f26fc43df0 🔥(frontend) remove DocTagPublic component
DocTagPublic component was removed because
it was not used.
2025-03-17 11:16:50 +01:00
Anthony LC
05a6818439 🧑‍💻(e2e) display more information when auth fails
When the auth fails, it was quite obscure to
understand what was going on.
We now take a screenshot of the page and display
the console logs.
2025-03-17 09:30:19 +01:00
Anthony LC
8056fd7d66 🚚(frontend) add import path for features/docs tsconfig
To simplify imports, we add the import path @/docs
to target ./src/features/docs/.
We changed all imports to use this path.
2025-03-17 09:30:19 +01:00
Samuel Paccoud - DINUM
c85224af42 📝(README) remove AGPL mention
This mention was confusing. We are only using min.io for development
purposes and this has nothing to do with the project's licence.
2025-03-16 19:00:14 +01:00
Anthony LC
70f1b6a8e8 🚩(frontend) add feature flag on "Copy as HTML"
As a blue print, we add a feature flag on
"Copy as HTML" button in the doc toolbox.
This feature flag is controlled by the `CopyAsHtml`
feature flag.

Be aware:
- if the feature flag is disabled, the button
will be shown
- if the feature flag is enabled and send true,
the button will be shown
- if the feature flag is enabled and send false,
the button will be hidden
2025-03-14 16:26:12 +01:00
Anthony LC
0f07fdcb65 📈(frontend) get user analytics
We will identify users by their email address.
This will help us understand how users interact
with the platform and improve the user experience.
2025-03-14 16:26:12 +01:00
Anthony LC
2e13dfb9bc 📈(frontend) abstract analytics classes
Add abstract classes for analytics services.
We will be able to add easily any analytic
services.
Our first analytic service usecase is Posthog.
2025-03-14 16:26:12 +01:00
renovate[bot]
a026435eb7 ⬆️(dependencies) update canvg to v4.0.3 [SECURITY] 2025-03-14 15:29:27 +01:00
Anthony LC
7007d56c38 🐛(frontend) fix svg not rendering export dox
The svg was not rendering in the dox export.
We overwrite the default mapping to convert the
svg to png before rendering.
The images could be out of the page as well,
we fixed this issue by adding a maxWidth to the image.
2025-03-14 15:13:51 +01:00
Anthony LC
0405e6a3f6 🐛(frontend) fix svg not rendering export pdf
The svg was not rendering in the pdf export.
We overwrite the default mapping to convert the
svg to png before rendering.
The images could be out of the page as well,
we fixed this issue by adding a maxWidth to the image.
2025-03-14 15:13:51 +01:00
Anthony LC
cb8bd4b937 🐛(frontend) fix flakiness e2e tests title doc
- fix multiple states title
- wait for stabilize network after create
- fix test other chromium browsers
- improve grid delete test
2025-03-14 14:49:17 +01:00
Anthony LC
4316b4e67d (frontend) adapt export to divider block
We have a new block type, the divider block.
We have to adapt the export to handle this
new block type.
2025-03-14 10:39:07 +01:00
Anthony LC
534085439f (frontend) add divider blocks to the editor
Add a custom block to add a divider in the editor.
2025-03-14 10:39:07 +01:00
Anthony LC
da02d3d756 🎨(frontend) use blockquote tag for quote block
Use the blockquote tag for quote block instead of
a paragraph tag.
2025-03-13 16:38:33 +01:00
Anthony LC
87960d3773 (AI) add emojify action to ai transform
The emojify action add emojis to the important
parts of the text.
2025-03-13 16:38:33 +01:00
Anthony LC
e0af6d36e1 (AI) add beautify action to ai transform
The beautify action add emojis to the important
parts of the text and add formatting to the text
to make it more readable.
2025-03-13 16:38:33 +01:00
Anthony LC
cbf9091d1c ️(AI) improve formating of ai translation
The ai translation were quite lossy about formatting.
Colors, background, breaklines, table sizes were
lost in the translation.
We improve the AI translation request to keep
the formatting as close as possible by using
html instead of markdown.
2025-03-13 16:38:33 +01:00
Anthony LC
9176328200 ♻️(frontend) replace cors proxy for export
We were using the cors proxy of Blocknote.js
to export the document. Now we use our own proxy
to avoid CORS issues.
2025-03-13 12:39:32 +01:00
Anthony LC
6efc2377fe (back) create a cors proxy fetching docs external resources
When exporting a document in PDF and if the doc contains external
resources, we want to fetch them using a proxy bypassing CORS
restrictions. To ensure this endpoint is not used for something else
than fetching urls contains in the doc, we use access control and check
if the url really exists in the document.
2025-03-13 12:39:32 +01:00
Anthony LC
1c02b0ad8e 📝(frontend) change literal section open source
Change literal section open source.
2025-03-12 09:53:01 +01:00
Manuel Raynaud
007854a877 ️(back) use redis as session backend in developement
We want to persist the session during development. Otherwise the session
is reset everytime the server is restart. This behavior make developing
bot a front and back feature a nigthmare, we spend our time login again
and again
2025-03-12 08:28:20 +01:00
Anthony LC
57cead448d 🏷️(frontend) adapt new doc type nb_accesses_direct
We renamed the `nb_accesses` field to `nb_accesses_direct`
and added a new `nb_accesses_ancestors` field.
We adapt the frontend to use the new fields.
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
f20d256cd1 🐛(backend) fix numchild when soft deleting/restoring a document
The numchild attribute must be incremented/decremented manually
when we soft delete a document if we want it to remain accurate,
which is important to display the tree structure in the frontend.
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
76c01df3ae (backend) add number of direct accesses related to a document
The "nb_accesses" field was displaying the number of access instances
related to a document or any of its ancestors. Some features on the
frontend require to know how many of these access instances are related
to the document directly.
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
20315e9b60 (backend) limit link reach/role select options depending on ancestors
If a document already gets a link reach/role inheriting from one of its
ancestors, we should not propose setting link reach/role on the
document that would be more restrictive than what we inherited from
ancestors.
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
2203d49a52 (backend) add new "descendants" action to document API endpoint
We want to be able to make a search query inside a hierchical document.
It's elegant to do it as a document detail action so that we benefit
from access control.
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
56aa69f56a ♻️(backend) refactor list view to allow filtering other views
the "filter_queryset" method is called in the middle of the
"get_object" method. We use the "get_object" in actions like
"children", "tree", etc. which start by calling "get_object"
but return lists of documents.

We would like to apply filters to these views but the it didn't
work because the "get_object" method was also impacted by the
filters...

In a future PR, we should take control of the "get_object" method
and decouple all this. We need a quick solution to allow releasing
the hierchical documents feature in the frontend.
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
0aabf26694 (backend) add "tree" action on document API endpoint
We want to display the tree structure to which a document belongs
on the left side panel of its detail view. For this, we need an
endpoint to retrieve the list view of the document's ancestors
opened.

By opened, we mean that when display the document, we also need to
display its siblings. When displaying the parent of the current
document, we also need to display the siblings of the parent...
2025-03-11 09:32:48 +01:00
Samuel Paccoud - DINUM
fcf8b38021 (backend) allow forcing page size within limits
We want to be able to increase the page size with by passing
the query string parameter "page_size".
2025-03-11 09:32:48 +01:00
renovate[bot]
757d7f35cd ⬆️(dependencies) update django to v5.1.7 [SECURITY] 2025-03-10 10:19:56 +01:00
Anthony LC
fdc49dc002 🐛(frontend) remove scroll listener table content
During a useEffect cleaning, the selector was
not the correct one. The debounce was not being
removed correctly neither.
2025-03-10 10:06:59 +01:00
Anthony LC
197ba47f73 ⬆️(frontend) bump cunningham to 3.0.0
Last bump to react 19 was a breaking change with
the previous version of Cunnigham, so we need to
update cunningham to 3.0.0 to be compatible with it.
We can now remove Cunnigham from the list of ignored
dependencies in the renovate.json file.
2025-03-10 09:26:19 +01:00
Anthony LC
d5997ba9d5 ⬆️(frontend) bump to react 19.0.0
Last version of Blocknotes is compatible with
React 19.0.0, it seems even necessary to
bump the version of React to 19.0.0.
We bump the version of React to 19.0.0 and
remove the react packages from renovate
list of ignored dependencies.
2025-03-10 09:26:19 +01:00
Anthony LC
1c6d18fdf3 📌(frontend) pin yjs globally
We had a warning about yjs multiple versions
between dependencies. We pinned yjs globally
to avoid this warning and potential side effects.
2025-03-10 09:26:19 +01:00
Anthony LC
24d126f410 🚨(frontend) fix linter
- fix linter
- remove unnecessary style files
2025-03-10 09:26:19 +01:00
renovate[bot]
a5e1751cf3 ⬆️(dependencies) update js dependencies 2025-03-10 09:26:19 +01:00
Manuel Raynaud
0cabb655ad 🔒️(back) restrict access to favorite_list endpoint
favorite_list endpoint is accessible to anonymous user. This lead to an
error 500. This endpoint should be accessible only to authenticated
users.
2025-03-07 09:04:44 +01:00
Manuel Raynaud
38eb6d45b7 🐛(back) fix ValidationError exception handler
The exception handler for the ValidationError was not testing correctly
the existence of some attributes like `message_dict`.
2025-03-07 09:04:44 +01:00
Anthony LC
5bb7ad643a 🔖(minor) release 2.4.0
Added:
- (frontend) synchronize language-choice

Changed:
- Use sentry tags instead of extra scope

Fixed:
- 🐛(frontend) fix collaboration error
2025-03-06 15:59:34 +01:00
Anthony LC
57b8881fc6 📌(frontend) pin blocknote globally
Blocknote does not pinned the version.
We get bumped version instead of the version we want.
We pin the version of blocknote globally to
avoid this issue.
2025-03-06 15:09:17 +01:00
Anthony LC
89ad610ba6 🐛(frontend) fix collaboration error
We upgrade blocknote to 0.23.2-hotfix.0,
it includes a fix with the collaboration.
2025-03-06 15:09:17 +01:00
rvveber
251787b835 🚨(tests) add language related tests; fix getByRole not working in tests
- adds tests and test-utility for solid language switching in tests
- fixes where ...getByRole(menuitem... would not return a valid object
2025-03-05 14:29:24 +01:00
rvveber
f95173e096 🐛(frontend) allow left panel to update on language change
- fixes a bug where after language-sync the left panel would remain
  in the same language as before.
2025-03-05 14:29:24 +01:00
rvveber
a7944cce80 (app) get language from backend; set browser-detected language if null
- adds useLanguageSynchronizer hook to update the:
  1. frontend-language to the user-preference - if there is one.
  2. user-preference to the (browser-detected) frontend-language - otherwise.
2025-03-05 14:29:24 +01:00
rvveber
7941fc91d5 🐛(frontend) set toolbar-popup to current language
- ensure editor is translated to i18n.resolvedLanguage => en
  as i18n.language could hold more accurate locale => en-GB etc..
2025-03-05 14:29:24 +01:00
rvveber
7fc83a4fcd (frontend) the LanguagePicker now uses config as options
- config endpoint languages are used as available options for LanguagePicker
- updating the language from it, triggers an update on the user via API
2025-03-05 14:29:24 +01:00
rvveber
2bf47b7705 (backend) the LanguagePicker now uses config as options
- config endpoint languages are used as available options for LanguagePicker
- updating the language from it, triggers an update on the user via API
2025-03-05 14:29:24 +01:00
rvveber
23b0214a2a (frontend) add language utility for "locale"
- Adds a helper for working with locales
- More details in their annotations
- Unnecessary, if in the future, the backend uses
  the same locales as the keys in the translations (ISO 639-1)
2025-03-05 14:29:24 +01:00
rvveber
f244509de3 (frontend) add API access for 'language' attribute on User model
- allow the language attribute on the user to be updated via API
- add frontend function to update the user language via API
- extend defaults on the test users, to have fixed language in E2E tests
- extend types and variables using the types with the new field
2025-03-05 14:29:24 +01:00
rvveber
fda5f8f008 (backend) add API access for 'language' attribute on User model
- allow the language attribute on the user to be updated via API
- add frontend function to update the user language via API
- extend defaults on the test users, to have fixed language in E2E tests
- extend types and variables using the types with the new field
2025-03-05 14:29:24 +01:00
rvveber
9a79b09b07 🔨(backend) make the 'language' attribute on the User model nullable
- allow the language on the user to be unset
- set the default language to be unset
- helps us determine that the user has yet to set a language preference
2025-03-05 14:29:24 +01:00
rvveber
b24acd14e2 🔨(frontend) email invitation in invited user's language
- language for invitation emails => language saved on the invited user
- if invited user does not exist yet => language of the sending user
- if for some reason no sending user => system default language
2025-03-05 14:29:24 +01:00
rvveber
1531846115 🔨(backend) email invitation in invited user's language
- language for invitation emails => language saved on the invited user
- if invited user does not exist yet => language of the sending user
- if for some reason no sending user => system default language
2025-03-05 14:29:24 +01:00
Manuel Raynaud
ebf6d46e37 ♻️(front) use sentry tags instead of extra scope
To ease filtering issues on sentry, we want to use tags instead of extra
scope. Tags are indexed and searchable, it's not the case with extra
scope. Moreover using setEtra to add additional data is deprecated.
2025-03-05 10:26:23 +01:00
Manuel Raynaud
b9b5f86cf4 ️(back) restrict documents to restore using only the queryset
To determine the descendant to restore or not, we were looking building
a complex exclude clause. This can be simplify focusing only on data we
already have without making an extra query to fetch the list of
descendant to exclude.
2025-03-04 18:03:18 +01:00
renovate[bot]
56412b0be5 ⬆️(dependencies) update python dependencies 2025-03-04 14:24:58 +01:00
Anthony LC
af052cd06b 🔖(minor) release 2.3.0
Added:
- 💄(frontend) add error pages
- 🔒️ Manage unsafe attachments
- (frontend) Custom block quote with export
- (frontend) add open source section homepage

Changed:
- 🛂(frontend) Restore version visibility
- 📝(doc) minor README.md formatting and wording enhancements
- ♻️Stop setting a default title on doc creation
- ♻️(frontend) misc ui improvements

Fixed:
- 🐛(backend) allow any type of extensions for media download
- ♻️(frontend) improve table pdf rendering
2025-03-04 12:12:57 +01:00
Anthony LC
8927635c5f 💄(frontend) hide Crisp when mobile and modal
The Crisp button was hidding buttons on mobile
when a modal was open. This commit hides the
Crisp button when a modal is open on mobile.
2025-03-04 12:12:57 +01:00
Anthony LC
76bce4313b 🩹(frontend) fine tuning 2.3.0
- improve medium button style when 2 lines
- improve design on Firefox input title
- manage title modal without doc title
- improve redirect when 401
2025-03-04 12:12:57 +01:00
Anthony LC
5ac71bfac1 🐛(service-worker) update sw to create a doc without body
Offline creation of a doc was broken because we
don't add a default title anymore when we create a
doc, leading to POST requests without body.
we need to adapt the service worker to handle this
case.
2025-03-04 12:12:57 +01:00
Anthony LC
cb4e148afc ♻️(email) adapt email when no title
Default title is not set when we create a document
anymore. We need to adapt the email to handle
this case.
2025-03-04 12:12:57 +01:00
AntoLC
2d24825be0 🌐(i18n) update translated strings
Update translated files with new translations
2025-03-04 12:12:57 +01:00
Anthony LC
7b1ddc0e05 🛂(backend) remove svg from unsafe
We added content-security-policy on nginx.
It should be safe to allow svg files now.
We remove the svg file from the unsafe
attachments list. We adapt the tests accordingly.
2025-03-03 13:18:40 +01:00
Manuel Raynaud
22a665e535 🔒️(nginx) manage Content-Security-Policy in nginx config
The media route is managed by nginx. On this route we want to add the
Content-Security-Header to forbid fetching any resources.
See : https://content-security-policy.com/
2025-03-03 13:18:40 +01:00
Manuel Raynaud
a22bf95bce 🔒️(back) set ContentDisposition on media upload
On the media upload endpoint, we want to set the content-disposition
header. Its value is based on the uploaded file mime-type and if flagged
as unsafe. If the file is not an image or is unsafe then the
contentDisposition is set to attachment to force its download.
Otherwise, we set it to inline.
2025-03-03 13:18:40 +01:00
Anthony LC
3ce1826355 🚚(frontend) toolbar components in BlockNoteToolBar folder
We moved the toolbar components in BlockNoteToolBar
folder.
2025-03-03 13:18:40 +01:00
Anthony LC
d099d58f77 🛂(frontend) secure download button
Blocknote download button opens the file in a new
tab, which could be not secure because of XSS attacks.
We replace the download button with a new one that
downloads the file instead of opening it in a new tab.
Some files are flags as unsafe (SVG / js / exe),
for these files we add a confirmation modal before
downloading the file to prevent the user from
downloading a file that could be harmful.
In the future, we could add other security layers
from this model, to analyze the file before
downloading it by example.
2025-03-03 13:18:40 +01:00
Anthony LC
ebd49f05a8 🚸(frontend) block click on unsafe image
We want to prevent the user to open unsafe images
in the browser. We blocked the click on the images.
To download them, the user will have to use the
download button.
2025-03-03 13:18:40 +01:00
Anthony LC
315c2c2c43 🐛(frontend) improve authenticated state
It can happen that the user is authenticated
then the token is expired. The authenticated
state should be updated to false in this case.
2025-03-03 13:18:40 +01:00
Anthony LC
e442908c50 💄(frontend) improve the design of the alert error
Since the new design implementation,
the alert error was not looking good.
This commit improves the design of the alert error.
2025-03-03 13:18:40 +01:00
Anthony LC
6672292d93 🛂(backend) add unsafe in the attachments filename
The frontend cannot access custom headers of a file,
so we need to add a flag in the filename.
We add the `unsafe` flag in the filename to
indicate that the file is unsafe.

Previous filename: "/{UUID4}.{extension}"
New filename: "/{UUID4}-unsafe.{extension}"
2025-03-03 13:18:40 +01:00
Manuel Raynaud
7dda74421f ♻️(back) extract ancestor deleted_at directly from db in restore method
In the restore method, all the ancestors with a deleted_at date set are
extracted from the database and then the oldest value is extracted using
the min python function. This usage of min can be removed by sorting
directly the deleted_at at the databse level and then fetching the first
one. It's faster and easier to maintain.
2025-03-03 13:05:36 +01:00
Anthony LC
9c25b684e3 (frontend) add open source section homepage
We decided to add the open source section on
the homepage of Docs.
2025-03-03 12:42:18 +01:00
Anthony LC
cd5ee3fb7c (frontend) adapt export to quote block
We have a new block type, the quote block.
We have to adapt the export to handle this
new block type.
2025-03-03 12:27:02 +01:00
Anthony LC
942c0f059c 🏗️(frontend) blockMapping refactoring
As made for TablePDF, we separate the block mapping
in separate files. This will allow us to have
a better separation of concerns and to have
a more maintainable codebase.
We improve as well the typing. It will be easier
to add new blocks in the future.
2025-03-03 12:27:02 +01:00
Anthony LC
3acee1e6fa (frontend) create feature doc-export
Create the feature doc-export, it will be
responsible for exporting the document.
2025-03-03 12:27:02 +01:00
Anthony LC
26ea32bd0b 🏷️(frontend) adapt title types
We recently changed the default title behavior.
It can now be undefined, we have to change the
types accordingly.
2025-03-03 12:27:02 +01:00
Anthony LC
7f6ffa0123 (frontend) add quote blocks to the editor
Add a custom block to quote in the editor.
2025-03-03 12:27:02 +01:00
Samuel Paccoud - DINUM
ef2127585c 🐛(backend) allow any type of extensions for media download
The regex to validate media file extensions was too restrictive.
2025-03-03 11:21:41 +01:00
Anthony LC
54a75bc338 ♻️(frontend) update some elements
- add panel information when document is
authenticated
- add a copy link button in the toolbox
on the document
- fix when long title document
- modals fit design
- mobile responsive changes
2025-02-24 10:49:20 +01:00
Anthony LC
50d098c777 💄(frontend) adapt language picker design
Adapt the language picker design to
to fit with the new design of the app.
2025-02-24 10:49:20 +01:00
Anthony LC
757c09b189 ♻️(frontend) adapt other error pages to new design
Adapt the other error pages to the new design.
2025-02-24 10:27:42 +01:00
Anthony LC
30c5cfab62 💄(frontend) add page 401
Add a 401 page when user try to access a
doc that need authentication.
2025-02-24 10:27:42 +01:00
Anthony LC
f069329e18 💄(frontend) add page 403
Add a page and integrate the design for the
403 error page.
2025-02-24 10:27:42 +01:00
Manuel Raynaud
ef8ee67553 ♻️(compose) set compose name in the docker-compose file
The helper bin/compose was using the option -p to set the compose
project name but this option is not used in the Makefile. This can lead
to different way to use the docker compose file definition with
different project name. In order to have a consistent name everywhere
and for everybody, we set the name in the docker compose file itself.
2025-02-20 10:32:37 +01:00
Anthony LC
ad47fc2d60 (e2e) global setup authentication
Because of the parallelism of the tests,
the authentication setup was flaky. Sometimes
the tests would run before the authentication
was complete.
We change to a global setup instead of the
project dependency setup, it should be more
reliable.
We improved the waiting states of the authentication
setup.
2025-02-19 14:57:15 +01:00
Nathan Vasse
009f5d6ed4 ♻️(front) improve table pdf rendering
The previous way of rendering table was causing issues when tables
could not fit on one page. I then came accross this discussion
https://github.com/diegomura/react-pdf/issues/2343. The author
created a lib to improve the rendering of table, it's better, but
still not perfect maybe.
2025-02-19 13:58:29 +01:00
Nathan Vasse
64d0072c8d 🐛(front) improve text rendering in pdf
The rendered text had unwanted line breaks in middle of them.
It was because we were not using the appropriate Text component, the
one to be used in the one from react-pdf.
2025-02-18 17:01:58 +01:00
Anthony LC
aefbc2e0b9 🛂(frontend) hide version restore button when reader
When you are a reader member, you have the right to
see the version but you cannot restore
them. So, we hide the restore button
when you are a reader.
2025-02-18 09:30:37 +01:00
Anthony LC
15dc1e3012 🦺(migration) add back the migration folders to linter
Previous commit add "core/tests/migrations".
The linter could not pass on it because all the
migration folders were excluded from the linter.
We remove this exclusion, tests and migrations can
now be linted and formatted automatically.
2025-02-15 23:39:17 +01:00
Anthony LC
6cc20aeacb 🩹(migration) add migration to update default titles
The frontend was setting a default titles for
documents with empty titles.
This migration updates the document table to set
the title to null instead of the default title.
We add a test to ensure that the migration
works as expected.
2025-02-15 23:39:17 +01:00
Anthony LC
7da7214afb (backend) add django-test-migrations
Add django-test-migrations to the project.
It is a tool that helps to test Django migrations.
2025-02-15 23:39:17 +01:00
Anthony LC
c369419512 ♻️(frontend) stop setting a default title on doc creation
We were setting a default title to our document
during creation, but we should not do that,
it created lot of similar titles, lot of
documents will show up during search.
2025-02-15 23:39:17 +01:00
Anthony LC
d9ad397c94 🩹(frontend) minor fixes
- fix linter warning on one e2e test
- improve logo svg
- improve cursor
- improve grid loader
2025-02-15 23:39:17 +01:00
Manuel Raynaud
3191d890f3 ♻️(docker) rename frontend-dev service in frontend
The frontend-dev service is in fact using the production image. We
rename it in frontend accordingly with what it really does. We also have
to change name rules in Makefile to be consistent.
2025-02-14 19:54:38 +01:00
Manuel Raynaud
68f3387539 ♻️(make) make run command starting everything
The run command is not starting the frontend application. We change the
run commands. The run command is strating everything. The run-backend
command is starting all services needed to use the backend application.
2025-02-14 19:54:38 +01:00
Manuel Raynaud
0dc8b4556c ♻️(docker) remove usage of dockerize
We remove dockerize and use healthcheck on docker compose services
instead.
2025-02-14 19:54:38 +01:00
Manuel Raynaud
e123e91959 🐛(nginx) increase nginx buffer size when proxifying keycloak
Nginx is used to proxify keycloak in our development configuration. When
a new user is created keycloak is send a large amount of headers in its
response and the default nginx config is not enough to handle this
amount of headers. We have to increase the proxy buffer size to handle
them.
2025-02-14 19:54:38 +01:00
Bastien Guerry
2709400773 🔊(changelog) add a changelog entry
Add "📝(doc) minor README.md formatting and wording enhancements" in
the unreleased section.
2025-02-14 17:39:52 +01:00
Bastien Guerry
8281c6159b 📝(doc) minor README.md formatting and wording enhancements
See https://github.com/suitenumerique/docs/issues/622
2025-02-14 17:39:52 +01:00
Anthony LC
296dbb7957 🔖(minor) release 2.2.0
Added:
- 📝(doc) Add security.md and codeofconduct.md
- (frontend) add home page
- (frontend) cursor display on activity
- (frontend) Add export page break

Changed:
- 🔧(backend) make AI feature reach configurable

Fixed:
- 🌐(CI) Fix email partially translated
- 🐛(frontend) fix cursor breakline
- 🐛(frontend) fix style pdf export
2025-02-11 14:16:58 +01:00
Anthony LC
3827f0f799 🩹(frontend) fine tuning v2.2.0
Fix minor issues:
- Add some height to the carret in the editor
- Improve css in mail templates
- Improve images resolution on homepage
2025-02-11 14:16:58 +01:00
Anthony LC
d89e3dc6d4 🛂(frontend) display the AI buttons depend abilities
Anybody with edit right could use the AI.
We changed this behavior, now we have to be
authentified with edit right.
We update the UI to display the AI buttons
only if the user has the correct AI ability.
2025-02-11 10:54:55 +01:00
Samuel Paccoud - DINUM
91cf5f9367 🔧(backend) make AI feature reach configurable
We want to be able to define whether AI features are available to
anonymous users who gained editor access on a document, or if we
demand that they be authenticated or even if we demand that they
gained their editor access via a specific document access.

Being authenticated is now the default value. This will change the
default behavior on your existing instance (see UPGRADE.md)
2025-02-11 10:54:55 +01:00
Anthony LC
5cc4b07cf6 💄(frontend) improve caret style
Improve the caret, to looks more like the
google doc caret.
2025-02-10 15:53:03 +01:00
Anthony LC
0cfc242e09 🚚(frontend) move Blocknote styles
Move Blocknote styles to a separate file.
2025-02-10 15:53:03 +01:00
Anthony LC
a6b3cfdb0c (frontend) add export page break
Blocknotejs introduced the ability to export a
document with page breaks.
This commit adds the page break feature to the
editor and so to our export feature.
2025-02-10 15:53:03 +01:00
Anthony LC
5ead18c94c 💬(frontend) add lacking buttons open source section
Some buttons were lacking in the open source
section of the home page. This commit adds them.
2025-02-10 15:40:23 +01:00
AntoLC
5eeb8cae5c 🌐(i18n) update translated strings
Update translated files with new translations
2025-02-10 15:40:23 +01:00
Jacques ROUSSEL
68bf024005 (helm) add pdbs to deployments
In order to avoid a service interruption during a Kubernetes (k8s)
upgrade, we add a Pod Disruption Budget (PDB) to deployments.
2025-02-10 13:05:54 +01:00
Anthony LC
fdd1068c90 (frontend) fix test copy html
The recent upgrade of Blocknote has caused the
test to fail.
This commit updates the test to reflect
the new html structure.
2025-02-10 10:50:29 +01:00
Anthony LC
ba695bf647 ⬇️(frontend) downgrade to @react-pdf/renderer to 4.1.6
The version 4.2.1 of @react-pdf/renderer
is not compatible @blocknote/xl-pdf-exporter
and @blocknote/xl-docx-exporter.
2025-02-10 10:50:29 +01:00
Anthony LC
27e7aec193 💄(frontend) improve styles export pdf
When exporting a document to PDF, the headings
spacings were too small, the break lines were
not displayed. This commit fixes these issues
by replacing the needed blocks.
2025-02-10 10:50:29 +01:00
Anthony LC
58b712a1de 🐛(frontend) fix cursor breakline
We had breakline issues with the initial
cursor because of some css properties.
We changed the cursor css to not take
any space in the lines,
avoiding the breakline issues.
We keep the new cursor visibility
feature (always, activity).
2025-02-10 10:50:29 +01:00
renovate[bot]
08f9036523 ⬆️(dependencies) update js dependencies 2025-02-10 10:50:29 +01:00
Anthony LC
ebe3efc8f7 💄(frontend) update the favicon
Update the favicon with a better one.
2025-02-07 18:18:22 +01:00
Anthony LC
66fbf27913 🩹(frontend) fix PageLayout
The page layout was rendered behind the header,
which caused the top of the mention legales pages
to be hidden.
This commit fixes this issue.
2025-02-07 18:18:22 +01:00
Anthony LC
20e4a4e42a 🥚(frontend) easter egg in dev tools
Add a easter egg in the browser dev tools.
2025-02-07 18:18:22 +01:00
Anthony LC
1aa4844eeb 🎨(frontend) add dsfr proconnect homepage
If we are with the DSFR theme, we need to add the
proconnect button to the homepage.
We add an option in the cunningham theme to
display the proconnect section instead of the
opensource section.
2025-02-07 18:18:22 +01:00
Nathan Panchout
4bb9c092cb (frontend) add white label homepage
Add white label homepage with new assets and
components. When the user is not logged in,
the homepage will be displayed.
2025-02-07 18:18:22 +01:00
Anthony LC
c493eb8924 ♻️(frontend) use a hook instead of a store for auth
We will use a hook instead of a store for the auth
feature. The hook will be powered by ReactQuery,
it will provide us fine-grained control over the
auth state and will be easier to use.
2025-02-07 18:18:22 +01:00
Anthony LC
40fdf97520 🚚(frontend) move auth to its own feature
We will move auth to its own feature to make it
easier to manage and to make it more modular.
2025-02-07 18:18:22 +01:00
Anthony LC
91b10e75dd 💄(email) fix line height email title
The line height of the email title was not
the correct size. We let the title managing
its own line height.
2025-02-07 17:05:49 +01:00
Anthony LC
7a6da10e1c 🐛(i18n) add back the missing email translations
We changed the way we upload the translations to
Crowdin, some translations were missing for the
email templates. We add them back and improve
the tests to make sure we don't forget them again.
2025-02-07 17:05:49 +01:00
Anthony LC
004e8ec645 🌐(CI) build mails when crowdin_upload workflow
When we were executing the crowdin_upload workflow,
we were not building the mail template to dispatch it
to the backend. It resulted in the mail not being
totally translated. This commit fixes that issue
by adding the build mail step to the crowdin_upload.
To do so, we added it to the dependencies workflow.
"dependencies" workflow is callable by other
workflows that need a specific job.
2025-02-07 17:05:49 +01:00
virgile-deville
a1bca9c436 📝(doc) add security.md and conduct.md policies
We need a safe way for people to report vulnerabilities.
People now can go on SECURITY.md and follow our policy.

We want to have a policy for expected behaviour.
People can check out CODE_OF_CONDUCT.md.
2025-02-02 14:53:29 +01:00
Sylvain Zimmer
d02fa1ddd4 ✏️(readme) fix a few typos
Found a few errors after proof-reading the README
2025-01-30 19:00:56 +01:00
Anthony LC
1fd66d3081 🔖(minor) release 2.1.0
Added:
- (backend) add soft delete and restore API endpoints to documents
- (backend) allow organizing documents in a tree structure
- (backend) add "excerpt" field to document list serializer
- (backend) add github actions to manage Crowdin workflow
- 📈Integrate Posthog
- 🏷️(backend) add content-type to uploaded file
- (frontend) export pdf docx front side7

Changed:
- 💄(frontend) add abilities on doc row
- 💄(frontend) improve DocsGridItem responsive padding
- 🔧(backend) Bump maximum page size to 200
- 📝(doc) Improve Read me

Fixed:
- 🐛Fix invitations

Removed:
- 🔥(backend) remove "content" field from list serializer
2025-01-30 12:50:44 +01:00
AntoLC
bae8c4c563 🌐(i18n) update translated strings
Update translated files with new translations
2025-01-30 11:37:05 +01:00
Anthony LC
0dae35dab1 ✏️(frontend) replace Word / Open Office by Docx
Replace the naming of the select export
options from Word / Open Office to Docx.
2025-01-30 11:37:05 +01:00
Anthony LC
929a50b573 ⬇️(frontend) downgrade cunningham to 2.9.4
The last version of Cunningham has problems.
Better to downgrade to the previous version.
We add cunningham library to renovate.json
to prevent future upgrade with Renovate.
An issue will be open to upgrade to the
last version manually.
2025-01-29 17:45:14 +01:00
Anthony LC
83dfd26d1c ⬇️(frontend) downgrade to react 18
We still have conflict with React 19, better to
downgrade to react 18 for the moment.
We add the react 18 libs to renovate.json
to prevent future upgrade with Renovate.
An issue will be open to upgrade to React 19
manually.
2025-01-29 17:45:14 +01:00
renovate[bot]
addc6a331f ⬆️(dependencies) update js dependencies 2025-01-29 17:45:14 +01:00
Samuel Paccoud - DINUM
5c5763a0ef 🗃️(backend) make document/template creation atomic
When creating a new document/template via the API, we add the
logged-in user as owner of the created object. This should be
done atomically with the object creation to make sure we don't
end-up with an orphan object that the creator can't access
anymore.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
5042f4ca47 ♻️(backend) generalize use of SerializerPerActionMixin in viewsets
We improve the serializer and generalize its use for clarity.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
8c247c8777 🔧(backend) bump maximum page size to 200
The new UI with infinite scroll calls for longer pages which we
are able to produce thanks to the many optimizations on the list
view.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
239342fbbd (backend) add API endpoint action to restore a soft deleted document
Only owners can see and restore deleted documents. They can only do
it during the grace period before the document is considered hard
deleted and hidden from everybody on the API.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
8ccfdb3c6a (backend) add soft delete to documents and refactor db queryset
Now that we have introduced a document tree structure, it is not
possible to allow deleting documents anymore as it impacts the whole
subtree below the deleted document and the consequences are too big.

We introduce soft delete in order to give a second thought to the
document's owner (who is the only one to be allowed to delete a
document). After a document is soft deleted, the owner can still
see it in the trashbin (/api/v1.0/documents/trashbin).
After a grace period (30 days be default) the document disappears
from the trashbin and can't be restored anymore. Note that even
then it is still kept in database. Cleaning the database to erase
deleted documents after the grace period can be done as a maintenance
script.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
4de03d292a (backend) add API endpoint to move a document in the document tree
Only administrators or owners of a document can move it to a target
document for which they are also administrator or owner.

We allow different moving modes:
- first-child: move the document as the first child of the target
- last-child: move the document as the last child of the target
- first-sibling: move the document as the first sibling of the target
- last-sibling: move the document as the last sibling of the target
- left: move the document as sibling ordered just before the target
- right: move the document as sibling ordered just after the target

The whole subtree below the document that is being moved, moves as
well and remains below the document after it is moved.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
2e8a399668 (backend) override defaults.NB_OBJECTS for the demo tests
Otherwise when you increase the number of objects to test how the
application scales, the demo test will take too long...
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
7b39b3f7f6 (backend) improve test error
This test was missing the status code check. Without this check
the error that follows does not make sense because the content
returned is not at all what we expect in the following assert
statement.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
0003f9d0de (backend) add user roles as field in the document API representation
user roles were already computed as an annotation on the query for
performance as we must look at all the document's ancestors to determine
the roles that apply recursively. We can easily expose them as readonly
via the serializer.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
8117866ce7 ♻️(backend) remove content from list serializer and introduce excerpt
Including the content field in the list view is not efficient as we need
to query the object storage to retrieve it. We want to display an excerpt
of the content on the list view so we should store it in database. We
let the frontend compute it and save it for us in the new "excerpt" field
because we are not supposed to have access to the content (E2EE feature coming)
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
1d0386d9b5 (backend) add API endpoint to create children for a document
We add a POST method to the existing children endpoint.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
7ff4bc457f (backend) add API endpoint to list a document's children
This endpoint is nested under a document's detail endpoint.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
4333b46901 (backend) add depth, path and numchild to serialized document
This information is useful for the frontend to display the document
tree structure and is cheap to expose.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
d073a9c9b3 (backend) retrieve & update a document taking into account ancestors
A document should inherit the access rights a user has on any of its
ancestors.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
48662ceecb (backend) list only the first visible parent document for a user
Now that we have a tree structure, we should only include parents of
a visible subtree in list results.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
9a12452c26 🚚(backend) split test files to make place for tests on tree structure
The test_api_documents_list file was getting too long. We can extract
tests on filters and ordering.
2025-01-29 14:39:47 +01:00
Samuel Paccoud - DINUM
276b4f7c1b (backend) add django-treebeard to allow tree structure on documents
We choose to use Django-treebeard for its quality, performance and
stability. Adding tree structure to documents is as simple as
inheriting from the MP_Node class.
2025-01-29 14:39:47 +01:00
Anthony LC
0189078917 (CI) fix test-e2e-other-browser job
A recent update to the impress-frontend workflow
caused the test-e2e-other-browser job to fail.
This commit fixes the issue.
2025-01-29 14:23:46 +01:00
Anthony LC
6569f61fc4 (e2e) fix flakiness
Some flakiness appeared in the e2e tests. It
started to impact many pull requests. Time to
fix them.
2025-01-29 14:23:46 +01:00
Nathan Panchout
7880391648 ️(frontend) rollback test changes
Changes were pushed by mistake. We go back. We reorder the tests
correctly and return the environment variables to their original state.
2025-01-29 12:16:10 +01:00
Nathan Panchout
9b95a9c551 🚸(frontend) prevent duplicate invite user row in DocShareModal
Modify the DocShareModal to avoid showing the invite user row when the
email already exists in the search results, preventing redundant invite
options
2025-01-29 12:16:10 +01:00
Manuel Raynaud
3b151cf580 🍱(readme) update europe <3 opensource logo name
The name used for the logo europe<3opensource was causing trouble on
windows systems. We have to rename it with a more "normal" name.
2025-01-29 12:16:10 +01:00
virgile-deville
8b0f4db650 📝(doc) improve readme.md after v2 update
We want to serve as an example of our open source doc best practices.
We want people to find out.
- Which libraries we support
- How they can contribute translations and code
2025-01-28 20:26:40 +01:00
Nathan Panchout
a39990d90f 🚸(frontend) simplify invite user row logic in DocShareModal
Refactor the endActions logic to show invite user row when searching by
email, removing the unnecessary length check for users
2025-01-28 18:15:18 +01:00
Samuel Paccoud - DINUM
609ff91894 🚸(backend) on user search match emails by Levenstein distance
When the query looks like an email (includes @) we search by
Levenstein distance because we are just trying to prevent typing
errors, not searching anymore.

It is important to still propose results with a short Levenstein
distance because it is frequent to forget a double letter in
someone's name for example "Pacoud" or even "pacou" instead of
"Paccoud" and we want to prevent duplicates or failing on
invitation.

We consider the query string to be an email as soon as it contains
a "@" character. Trying harder to identify a string that is really
an email would lead to weird behaviors like toto@example.gouv looking
like and email but if we continue typing toto@example.gouv.f not
looking like an email... before toto@example.gouv.fr finally looking
like an email. The result would be jumping from one type of search
to the other. As soon as there is a "@" in the query, we can be
sure that the user is not looking for a name anymore and we can
switch to matching by Levenstein distance.
2025-01-28 18:15:18 +01:00
Manuel Raynaud
265a24fe7e 🚨(back) update code linting with ruff
Apply new rules provided with ruff version v0.9.3
2025-01-28 16:23:07 +01:00
renovate[bot]
9bc2b4877f ⬆️(dependencies) update python dependencies 2025-01-28 16:23:07 +01:00
Nathan Panchout
b93b43abe8 💄(frontend) improve DocsGridItem responsive padding
- Adjusted padding and alignment for desktop and mobile views
- Conditionally applied CSS styles based on screen size
2025-01-28 13:59:22 +01:00
Anthony LC
dd8bb18f69 🔊(changelog) add some changelog entries
Add some changelog entries that can be useful to
display in the release notes.
2025-01-28 13:36:03 +01:00
Anthony LC
545e8b2a3c 🔥(backend) remove code related to export pdf docx
The export is managed by the frontend, so we
don't need the code related to the export
in the backend side anymore.
2025-01-28 13:36:03 +01:00
Anthony LC
81837aff2b (frontend) export pdf docx front side
We have added the export to pdf and docx feature
to the front side. Thanks to that, the images are now
correctly exported even when the doc is private.
To be able to export the doc, the data must be
in blocknote format, for legacy purpose, we have
to convert the template to blocknote format before
exporting it.
2025-01-28 13:36:03 +01:00
lunika
40c1107959 🌐(i18n) update translated strings
Update translated files with new translations
2025-01-28 12:59:54 +01:00
Manuel Raynaud
0d7d42254b (helm) add a job allowing to run arbitrary management command
For a specific deployment we may need to run a specific management
command, like the one added previously updating all files content-type.
A template is added responsible to manage this case. The job will be
created only if the backend.job.command is set.
2025-01-28 10:33:30 +01:00
Anthony LC
67dc7feb98 🚑️(backend) command to update attachment content-type
The uploaded files in the system are missing
the content-type.
We add a command to update the content-type of
the existing uploaded files.
This command will run one time when we will deploy
to the environments.
2025-01-28 10:33:30 +01:00
Anthony LC
5b4b100e90 🏷️(backend) add content-type to uploaded files
All the uploaded files had the content-type set
to `application/octet-stream`. It create issues
when the file is downloaded from the frontend
because the browser doesn't know how to handle
the file.
We now determine the content-type of the file
and set it to the file object.
2025-01-28 10:33:30 +01:00
Anthony LC
b8be010389 🚚(helm) add posthog proxy
To contourn ads blocker, we add a proxy to the
posthog service. This way, we can access the
service from the same domain as the frontend.
2025-01-28 10:05:37 +01:00
Anthony LC
97cfa2c1ad (frontend) integrate posthog analytics
We integrate posthog, it will help us to track
user behavior and improve the product.
We get the configuration from the backend config
endpoint.
2025-01-28 10:05:37 +01:00
Anthony LC
c018c6fcf5 🔧(backend) add posthog configuration
We add the posthog configuration to the project.
We will expose the posthog configuration to the
frontend.
2025-01-28 10:05:37 +01:00
Nathan Panchout
70048328d1 (frontend) update document sharing tests
- Added Share button interactions in various document visibility
scenarios
- Updated test assertions for share and copy link functionality
- Improved test coverage for document sharing features
2025-01-28 08:59:28 +01:00
Nathan Panchout
55ddfe9181 💄(frontend) refactor document sharing and grid components
Improvements:
- Added disabled state for dropdown menus in share settings
- Updated document grid layout and responsiveness
- Simplified sharing and access count logic
- Improved tooltips and visibility of shared documents
- Created a new responsive doc grid hook
2025-01-28 08:59:28 +01:00
renovate[bot]
ee41d156c7 ⬆️(dependencies) update django to v5.1.5 [SECURITY] 2025-01-27 10:06:16 +01:00
Manuel Raynaud
5be2bc7360 ♻️(actions) create a reusable workflow to install front dependencies
In more than one workflow we need to install frontend dependencies and
this 3 workflows we are copy/pasting the same code. We want to refactor
this by creating a reusable workflow
https://docs.github.com/en/actions/sharing-automations/reusing-workflows
2025-01-24 12:22:48 +01:00
Manuel Raynaud
e46ba4f506 🌐(back) update source translations
In order to have a repo correctly confiogured to managed translation
with crowdin, source translations for the backend app are updated
2025-01-24 12:22:48 +01:00
Manuel Raynaud
7c8b969fa9 🔥(back) remove compiled translation file
The compiles translation file should not be track with git.
2025-01-24 12:22:48 +01:00
Manuel Raynaud
95515fd460 🌐(action) create a workflow to download translation
A new workflow is added responsible todownload new translated strings
and then create a pull request to update the code.
2025-01-24 12:22:48 +01:00
Manuel Raynaud
ce6cfc22ef 🌐(action) upload sources translation on crowdin
Crowdin has released its own github action to automatize translation
workflow. We want to use to upload sources when a PR is merged.
2025-01-24 12:22:48 +01:00
virgile-dev
4b3b441fc3 📝(project) update readme following upgrade to v2
Remove old mentions to "impress" following the repository renaming.
Improve and update descriptions to better reflect the status of the
project after release version 2.
2025-01-21 23:39:22 +01:00
Anthony LC
9194bf5a90 🔖(patch) release 2.0.1
Fixed:
🐛(frontend) title copy break app
2025-01-17 11:58:55 +01:00
Anthony LC
dc63a5839e ⬇️(frontend) downgraded blocknote to 0.21.0
The last version of Blocknote (0.22.0) has a bug,
when we copy paste a title, the app sometimes crashes.
Better to downgrade to 0.21.0 until the bug is fixed.
2025-01-17 11:33:47 +01:00
Anthony LC
d406846986 🎨(frontend) format css blocknote editor
We use the "css" function of style components
to format correctly the blocknote editor css.
2025-01-17 11:33:47 +01:00
Nathan Panchout
e85b07021e 🐛(frontend) fix collaboration cursor
- The collaboration slider is not fully shown when a user is at the very
top of the document
2025-01-17 11:02:41 +01:00
Nathan Panchout
282200ac3d 🐛(frontend) hide the sharing method when you don't have the rights
- Added a new hook `useCopyDocLink` to handle copying document links to
the clipboard with success/error notifications.
- Updated the `DocToolBox`, `DocsGridActions`, and `DocShareModal`
components to utilize the new copy link feature.
- Enhanced tests to verify the functionality of the copy link button in
various scenarios.
- Adjusted visibility checks for sharing options based on user access
rights.
2025-01-17 11:02:41 +01:00
Anthony LC
de8dea20d5 🔖(major) release 2.0.0
Added:
- 🔧(backend) add option to configure list of
essential OIDC claims
- 🔧(helm) add option to disable default tls
setting by @dominikkaminski
- 💄(frontend) Add left panel
- 💄(frontend) add filtering to left panel
- (frontend) new share modal ui
- (frontend) add favorite feature

Changed:
- 🏗️(yjs-server) organize yjs server
- ♻️(frontend) better separation collaboration
process
- 💄(frontend) updating the header and leftpanel
for responsive
- 💄(frontend) update DocsGrid component
- 💄(frontend) update DocsGridOptions component
- 💄(frontend) update DocHeader ui
- 💄(frontend) update doc versioning ui
- 💄(frontend) update doc summary u

Fixed:
- 🐛(backend) fix create document via s2s
if sub unknown but email found
- 🐛(frontend) hide search and create doc
button if not authenticated
- 🐛(backend) race condition creation issue
2025-01-15 12:46:00 +01:00
424 changed files with 37505 additions and 22036 deletions

77
.github/workflows/crowdin_download.yml vendored Normal file
View File

@@ -0,0 +1,77 @@
name: Download translations from Crowdin
on:
workflow_dispatch:
push:
branches:
- 'release/**'
jobs:
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
node_version: '20.x'
with-front-dependencies-installation: true
synchronize-with-crowdin:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Create empty source files
run: |
touch src/backend/locale/django.pot
mkdir -p src/frontend/packages/i18n/locales/impress/
touch src/frontend/packages/i18n/locales/impress/translations-crowdin.json
# crowdin workflow
- name: crowdin action
uses: crowdin/github-action@v2
with:
config: crowdin/config.yml
upload_sources: false
upload_translations: false
download_translations: true
create_pull_request: false
push_translations: false
push_sources: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# A numeric ID, found at https://crowdin.com/project/<projectName>/tools/api
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
# Visit https://crowdin.com/settings#api-key to create this token
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
CROWDIN_BASE_PATH: "../src/"
# frontend i18n
- name: Restore the frontend cache
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: generate translations files
working-directory: src/frontend
run: yarn i18n:deploy
# Create a new PR
- name: Create a new Pull Request with new translated strings
uses: peter-evans/create-pull-request@v7
with:
commit-message: |
🌐(i18n) update translated strings
Update translated files with new translations
title: 🌐(i18n) update translated strings
body: |
## Purpose
update translated strings
## Proposal
- [x] update translated strings
branch: i18n/update-translations
labels: i18n

76
.github/workflows/crowdin_upload.yml vendored Normal file
View File

@@ -0,0 +1,76 @@
name: Update crowdin sources
on:
workflow_dispatch:
push:
branches:
- main
jobs:
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
node_version: '20.x'
with-front-dependencies-installation: true
with-build_mails: true
synchronize-with-crowdin:
needs: install-dependencies
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
# Backend i18n
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: "3.12.6"
- name: Upgrade pip and setuptools
run: pip install --upgrade pip setuptools
- name: Install development dependencies
run: pip install --user .
working-directory: src/backend
- name: Restore the mail templates
uses: actions/cache@v4
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
fail-on-cache-miss: true
- name: Install gettext
run: |
sudo apt-get update
sudo apt-get install -y gettext pandoc
- name: generate pot files
working-directory: src/backend
run: |
DJANGO_CONFIGURATION=Build python manage.py makemessages -a --keep-pot
# frontend i18n
- name: Restore the frontend cache
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: generate source translation file
working-directory: src/frontend
run: yarn i18n:extract
# crowdin workflow
- name: crowdin action
uses: crowdin/github-action@v2
with:
config: crowdin/config.yml
upload_sources: true
upload_translations: false
download_translations: false
create_pull_request: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# A numeric ID, found at https://crowdin.com/project/<projectName>/tools/api
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
# Visit https://crowdin.com/settings#api-key to create this token
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
CROWDIN_BASE_PATH: "../src/"

85
.github/workflows/dependencies.yml vendored Normal file
View File

@@ -0,0 +1,85 @@
name: Dependency reusable workflow
on:
workflow_call:
inputs:
node_version:
required: false
default: '20.x'
type: string
with-front-dependencies-installation:
type: boolean
default: false
with-build_mails:
type: boolean
default: false
jobs:
front-dependencies-installation:
if: ${{ inputs.with-front-dependencies-installation == true }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Setup Node.js
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node_version }}
- name: Install dependencies
if: steps.front-node_modules.outputs.cache-hit != 'true'
run: cd src/frontend/ && yarn install --frozen-lockfile
- name: Cache install frontend
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
build-mails:
if: ${{ inputs.with-build_mails == true }}
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/mail
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore the mail templates
uses: actions/cache@v4
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
- name: Setup Node.js
if: steps.mail-templates.outputs.cache-hit != 'true'
uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node_version }}
- name: Install yarn
if: steps.mail-templates.outputs.cache-hit != 'true'
run: npm install -g yarn
- name: Install node dependencies
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn install --frozen-lockfile
- name: Build mails
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn build
- name: Cache mail templates
if: steps.mail-templates.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}

View File

@@ -125,8 +125,7 @@ jobs:
- build-and-push-frontend
- build-and-push-backend
runs-on: ubuntu-latest
if: |
github.event_name != 'pull_request'
if: github.event_name != 'pull_request'
steps:
-
name: Checkout repository
@@ -134,6 +133,6 @@ jobs:
-
name: Call argocd github webhook
run: |
data='{"ref": "'$GITHUB_REF'","repository": {"html_url":"'$GITHUB_SERVER_URL'/'$GITHUB_REPOSITORY'"}}'
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac ''${{ secrets.ARGOCD_PREPROD_WEBHOOK_SECRET}}'' | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" ${{ vars.ARGOCD_PREPROD_WEBHOOK_URL }}
data='{"ref": "'$GITHUB_REF'","repository": {"html_url":"'$GITHUB_SERVER_URL'/${{ secrets.DEPLOYMENT_REPO_URL }}"}}'
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac "${{ secrets.ARGOCD_PREPROD_WEBHOOK_SECRET }}" | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" ${{ vars.ARGOCD_PREPROD_WEBHOOK_URL }}

View File

@@ -11,7 +11,7 @@ jobs:
helmfile-lint:
runs-on: ubuntu-latest
container:
image: ghcr.io/helmfile/helmfile:latest
image: ghcr.io/helmfile/helmfile:v0.171.0
steps:
-
name: Checkout repository
@@ -22,9 +22,9 @@ jobs:
run: |
set -e
HELMFILE=src/helm/helmfile.yaml
environments=$(awk '/environments:/ {flag=1; next} flag && NF {print} !NF {flag=0}' "$HELMFILE" | grep -E '^[[:space:]]{2}[a-zA-Z]+' | sed 's/^[[:space:]]*//;s/:.*//')
environments=$(awk 'BEGIN {in_env=0} /^environments:/ {in_env=1; next} /^---/ {in_env=0} in_env && /^ [^ ]/ {gsub(/^ /,""); gsub(/:.*$/,""); print}' "$HELMFILE")
for env in $environments; do
echo "################### $env lint ###################"
helmfile -e $env -f $HELMFILE lint || exit 1
echo -e "\n"
done
done

View File

@@ -9,39 +9,16 @@ on:
- "*"
jobs:
install-front:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20.x"
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Install dependencies
if: steps.front-node_modules.outputs.cache-hit != 'true'
run: cd src/frontend/ && yarn install --frozen-lockfile
- name: Cache install frontend
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
node_version: '20.x'
with-front-dependencies-installation: true
test-front:
needs: install-dependencies
runs-on: ubuntu-latest
needs: install-front
steps:
- name: Checkout repository
uses: actions/checkout@v4
@@ -53,44 +30,54 @@ jobs:
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: Test App
run: cd src/frontend/ && yarn test
lint-front:
runs-on: ubuntu-latest
needs: install-front
needs: install-dependencies
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20.x"
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: Check linting
run: cd src/frontend/ && yarn lint
test-e2e-chromium:
runs-on: ubuntu-latest
needs: install-dependencies
timeout-minutes: 20
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20.x"
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: Set e2e env variables
run: cat env.d/development/common.e2e.dist >> env.d/development/common.dist
@@ -101,28 +88,6 @@ jobs:
- name: Start Docker services
run: make bootstrap FLUSH_ARGS='--no-input' cache=
# Tool to wait for a service to be ready
- name: Install Dockerize
run: |
curl -sSL https://github.com/jwilder/dockerize/releases/download/v0.8.0/dockerize-linux-amd64-v0.8.0.tar.gz | sudo tar -C /usr/local/bin -xzv
- name: Wait for services to be ready
run: |
printf "Minio check...\n"
dockerize -wait tcp://localhost:9000 -timeout 20s
printf "Keyclock check...\n"
dockerize -wait tcp://localhost:8080 -timeout 20s
printf "Server collaboration check...\n"
dockerize -wait tcp://localhost:4444 -timeout 20s
printf "Ngnix check...\n"
dockerize -wait tcp://localhost:8083 -timeout 20s
printf "DRF check...\n"
dockerize -wait tcp://localhost:8071 -timeout 20s
printf "Postgres Keyclock check...\n"
dockerize -wait tcp://localhost:5433 -timeout 20s
printf "Postgres back check...\n"
dockerize -wait tcp://localhost:15432 -timeout 20s
- name: Run e2e tests
run: cd src/frontend/ && yarn e2e:test --project='chromium'
@@ -141,12 +106,17 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20.x"
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: Set e2e env variables
run: cat env.d/development/common.e2e.dist >> env.d/development/common.dist

View File

@@ -9,6 +9,11 @@ on:
- "*"
jobs:
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
with-build_mails: true
lint-git:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request' # Makes sense only for pull requests
@@ -56,46 +61,6 @@ jobs:
exit 1
fi
build-mails:
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/mail
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: "18"
- name: Restore the mail templates
uses: actions/cache@v4
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
- name: Install yarn
if: steps.mail-templates.outputs.cache-hit != 'true'
run: npm install -g yarn
- name: Install node dependencies
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn install --frozen-lockfile
- name: Build mails
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn build
- name: Cache mail templates
if: steps.mail-templates.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
lint-back:
runs-on: ubuntu-latest
defaults:
@@ -121,7 +86,7 @@ jobs:
test-back:
runs-on: ubuntu-latest
needs: build-mails
needs: install-dependencies
defaults:
run:
@@ -169,6 +134,7 @@ jobs:
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
fail-on-cache-miss: true
- name: Start MinIO
run: |
@@ -206,10 +172,11 @@ jobs:
- name: Install development dependencies
run: pip install --user .[dev]
- name: Install gettext (required to compile messages)
- name: Install gettext (required to compile messages) and MIME support
run: |
sudo apt-get update
sudo apt-get install -y gettext pandoc
sudo apt-get install -y gettext pandoc shared-mime-info
sudo wget https://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/conf/mime.types -O /etc/mime.types
- name: Generate a MO file from strings extracted from the project
run: python manage.py compilemessages

1
.gitignore vendored
View File

@@ -30,6 +30,7 @@ MANIFEST
.next/
# Translations # Translations
*.mo
*.pot
# Environments

View File

@@ -6,17 +6,174 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0),
and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## Added
- 📄(legal) Require contributors to sign a DCO #779
## Changed
- ♻️(frontend) Integrate UI kit #783
## [2.6.0] - 2025-03-21
## Added
- 📝(doc) add publiccode.yml #770
## Changed
- 🚸(frontend) ctrl+k modal not when editor is focused #712
## Fixed
- 🐛(back) allow only images to be used with the cors-proxy #781
- 🐛(backend) stop returning inactive users on the list endpoint #636
- 🔒️(backend) require at least 5 characters to search for users #636
- 🔒️(back) throttle user list endpoint #636
- 🔒️(back) remove pagination and limit to 5 for user list endpoint #636
## [2.5.0] - 2025-03-18
## Added
- 📝(doc) Added GNU Make link to README #750
- ✨(frontend) add pinning on doc detail #711
- 🚩(frontend) feature flag analytic on copy as html #649
- ✨(frontend) Custom block divider with export #698
- 🌐(i18n) activate dutch language #742
- ✨(frontend) add Beautify action to AI transform #478
- ✨(frontend) add Emojify action to AI transform #478
## Changed
- 🧑‍💻(frontend) change literal section open source #702
- ♻️(frontend) replace cors proxy for export #695
- 🚨(gitlint) Allow uppercase in commit messages #756
- ♻️(frontend) Improve AI translations #478
## Fixed
- 🐛(frontend) SVG export #706
- 🐛(frontend) remove scroll listener table content #688
- 🔒️(back) restrict access to favorite_list endpoint #690
- 🐛(backend) refactor to fix filtering on children
and descendants views #695
- 🐛(action) fix notify-argocd workflow #713
- 🚨(helm) fix helmfile lint #736
- 🚚(frontend) redirect to 401 page when 401 error #759
## [2.4.0] - 2025-03-06
## Added
- ✨(frontend) synchronize language-choice #401
## Changed
- Use sentry tags instead of extra scope
## Fixed
- 🐛(frontend) fix collaboration error #684
## [2.3.0] - 2025-03-03
## Added
- ✨(backend) limit link reach/role select options depending on ancestors #645
- ✨(backend) add new "descendants" action to document API endpoint #645
- ✨(backend) new "tree" action on document detail endpoint #645
- ✨(backend) allow forcing page size within limits #645
- 💄(frontend) add error pages #643
- 🔒️ Manage unsafe attachments #663
- ✨(frontend) Custom block quote with export #646
- ✨(frontend) add open source section homepage #666
- ✨(frontend) synchronize language-choice #401
## Changed
- 🛂(frontend) Restore version visibility #629
- 📝(doc) minor README.md formatting and wording enhancements
-Stop setting a default title on doc creation #634
- ♻️(frontend) misc ui improvements #644
## Fixed
- 🐛(backend) allow any type of extensions for media download #671
- ♻️(frontend) improve table pdf rendering
- 🐛(email) invitation emails in receivers language
## [2.2.0] - 2025-02-10
## Added
- 📝(doc) Add security.md and codeofconduct.md #604
- ✨(frontend) add home page #608
- ✨(frontend) cursor display on activity #609
- ✨(frontend) Add export page break #623
## Changed
- 🔧(backend) make AI feature reach configurable #628
## Fixed
- 🌐(CI) Fix email partially translated #616
- 🐛(frontend) fix cursor breakline #609
- 🐛(frontend) fix style pdf export #609
## [2.1.0] - 2025-01-29
## Added
- ✨(backend) add soft delete and restore API endpoints to documents #516
- ✨(backend) allow organizing documents in a tree structure #516
- ✨(backend) add "excerpt" field to document list serializer #516
- ✨(backend) add github actions to manage Crowdin workflow #559 & #563
- 📈Integrate Posthog #540
- 🏷️(backend) add content-type to uploaded files #552
- ✨(frontend) export pdf docx front side #537
## Changed
- 💄(frontend) add abilities on doc row #581
- 💄(frontend) improve DocsGridItem responsive padding #582
- 🔧(backend) Bump maximum page size to 200 #516
- 📝(doc) Improve Read me #558
## Fixed
- 🐛Fix invitations #575
## Removed
- 🔥(backend) remove "content" field from list serializer # 516
## [2.0.1] - 2025-01-17
## Fixed
-🐛(frontend) share modal is shown when you don't have the abilities #557
-🐛(frontend) title copy break app #564
## [2.0.0] - 2025-01-13
## Added
- 🔧(backend) add option to configure list of essential OIDC claims #525 & #531
- 🔧(helm) add option to disable default tls setting by @dominikkaminski #519
- 💄(frontend) Add left panel #420
- 💄(frontend) add filtering to left panel #475
- ✨(frontend) new share modal ui #489
- ✨(frontend) add favorite feature #515
- 📝(documentation) Documentation about self-hosted installation #530
- ✨(helm) helm versioning #530
## Changed
@@ -25,9 +182,10 @@ and this project adheres to
- 💄(frontend) updating the header and leftpanel for responsive #421
- 💄(frontend) update DocsGrid component #431
- 💄(frontend) update DocsGridOptions component #432
- 💄(frontend) update DocHeader ui #446
- 💄(frontend) update DocHeader ui #448
- 💄(frontend) update doc versioning ui #463
- 💄(frontend) update doc summary ui #473
- 📝(doc) update readme.md to match V2 changes #558 & #572
## Fixed
@@ -51,12 +209,11 @@ and this project adheres to
- ⚡️(e2e) reduce flakiness on e2e tests #511
## Fixed
- 🐛(frontend) update doc editor height #481
- 💄(frontend) add doc search #485
## [1.9.0] - 2024-12-11
## Added
@@ -78,21 +235,18 @@ and this project adheres to
- 🐛(frontend) Fix hidden menu on Firefox #468
- 🐛(backend) fix sanitize problem IA #490
## [1.8.2] - 2024-11-28
## Changed
- ♻️(SW) change strategy html caching #460
## [1.8.1] - 2024-11-27
## Fixed
- 🐛(frontend) link not clickable and flickering firefox #457
## [1.8.0] - 2024-11-25
## Added
@@ -121,7 +275,6 @@ and this project adheres to
- 🐛(frontend) users have view access when revoked #387
- 🐛(frontend) fix placeholder editable when double clicks #454
## [1.7.0] - 2024-10-24
## Added
@@ -149,7 +302,6 @@ and this project adheres to
- 🔥(helm) remove infra related codes #366
## [1.6.0] - 2024-10-17
## Added
@@ -172,7 +324,6 @@ and this project adheres to
- 🐛(backend) fix nginx docker container #340
- 🐛(frontend) fix copy paste firefox #353
## [1.5.1] - 2024-10-10
## Fixed
@@ -207,7 +358,6 @@ and this project adheres to
- 🔧(backend) fix configuration to avoid different ssl warning #297
- 🐛(frontend) fix editor break line not working #302
## [1.4.0] - 2024-09-17
## Added
@@ -228,7 +378,6 @@ and this project adheres to
- 🐛(backend) Fix forcing ID when creating a document via API endpoint #234
- 🐛 Rebuild frontend dev container from makefile #248
## [1.3.0] - 2024-09-05
## Added
@@ -253,7 +402,6 @@ and this project adheres to
- 🔥(frontend) remove saving modal #213
## [1.2.1] - 2024-08-23
## Changed
@@ -261,7 +409,6 @@ and this project adheres to
- ♻️ Change ordering docs datagrid #195
- 🔥(helm) use scaleway email #194
## [1.2.0] - 2024-08-22
## Added
@@ -285,14 +432,14 @@ and this project adheres to
- ⚡️(CI) only e2e chrome mandatory #177
## Removed
- 🔥(helm) remove htaccess #181
- 🔥(helm) remove htaccess #181
## [1.1.0] - 2024-07-15
## Added
- 🤡(demo) generate dummy documents on dev users #120
- 🤡(demo) generate dummy documents on dev users #120
- ✨(frontend) create side modal component #134
- ✨(frontend) Doc grid actions (update / delete) #136
- ✨(frontend) Doc editor header information #137
@@ -303,12 +450,11 @@ and this project adheres to
- ♻️(frontend) create a doc from a modal #132
- ♻️(frontend) manage members from the share modal #140
## [1.0.0] - 2024-07-02
## Added
- 🛂(frontend) Manage the document's right (#75)
- 🛂(frontend) Manage the document's right (#75)
- ✨(frontend) Update document (#68)
- ✨(frontend) Remove document (#68)
- 🐳(docker) dockerize dev frontend (#63)
@@ -342,7 +488,6 @@ and this project adheres to
- 💚(CI) Remove trigger workflow on push tags on CI (#68)
- 🔥(frontend) Remove coming soon page (#121)
## [0.1.0] - 2024-05-24
## Added
@@ -350,8 +495,15 @@ and this project adheres to
- ✨(frontend) Coming Soon page (#67)
- 🚀 Impress, project to manage your documents easily and collaboratively.
[unreleased]: https://github.com/numerique-gouv/impress/compare/v1.10.0...main
[unreleased]: https://github.com/numerique-gouv/impress/compare/v2.6.0...main
[v2.5.0]: https://github.com/numerique-gouv/impress/releases/v2.6.0
[v2.5.0]: https://github.com/numerique-gouv/impress/releases/v2.5.0
[v2.4.0]: https://github.com/numerique-gouv/impress/releases/v2.4.0
[v2.3.0]: https://github.com/numerique-gouv/impress/releases/v2.3.0
[v2.2.0]: https://github.com/numerique-gouv/impress/releases/v2.2.0
[v2.1.0]: https://github.com/numerique-gouv/impress/releases/v2.1.0
[v2.0.1]: https://github.com/numerique-gouv/impress/releases/v2.0.1
[v2.0.0]: https://github.com/numerique-gouv/impress/releases/v2.0.0
[v1.10.0]: https://github.com/numerique-gouv/impress/releases/v1.10.0
[v1.9.0]: https://github.com/numerique-gouv/impress/releases/v1.9.0
[v1.8.2]: https://github.com/numerique-gouv/impress/releases/v1.8.2

75
CODE_OF_CONDUCT.md Normal file
View File

@@ -0,0 +1,75 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our community include:
- Demonstrating empathy and kindness toward other people
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
- Focusing on what is best not just for us as individuals, but for the overall community
Examples of unacceptable behavior include:
- The use of sexualized language or imagery, and sexual attention or advances of any kind
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or email address, without their explicit permission
- Other conduct which could reasonably be considered inappropriate in a professional setting
## Enforcement Responsibilities
- Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.
- Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.
## Scope
- This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
## Enforcement
- Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at docs@numerique.gouv.fr.
- All complaints will be reviewed and investigated promptly and fairly.
- All community leaders are obligated to respect the privacy and security of the reporter of any incident.
## Enforcement Guidelines
- Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this
## Code of Conduct:
1. Correction
Community Impact: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
Consequence: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.
2. Warning
Community Impact: A violation through a single incident or series of actions.
Consequence: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.
3. Temporary Ban
Community Impact: A serious violation of community standards, including sustained inappropriate behavior.
Consequence: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
4. Permanent Ban
Community Impact: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
Consequence: A permanent ban from any sort of public interaction within the community.
Attribution
This Code of Conduct is adapted from the Contributor Covenant, version 2.1, available at https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.
Community Impact Guidelines were inspired by Mozilla's code of conduct enforcement ladder.
For answers to common questions about this code of conduct, see the FAQ at https://www.contributor-covenant.org/faq. Translations are available at https://www.contributor-covenant.org/translations.

View File

@@ -2,7 +2,16 @@
Thank you for taking the time to contribute! Please follow these guidelines to ensure a smooth and productive workflow. 🚀🚀🚀
To get started with the project, please refer to the [README.md](https://github.com/numerique-gouv/impress/blob/main/README.md) for detailed instructions.
To get started with the project, please refer to the [README.md](https://github.com/suitenumerique/docs/blob/main/README.md) for detailed instructions.
Contributors are required to sign off their commits with `git commit --sign-off`: this confirms that they have read and accepted the [Developer's Certificate of Origin 1.1](https://developercertificate.org/).
Please also check out our [dev handbook](https://suitenumerique.gitbook.io/handbook) to learn our best practices.
## Help us with translations
You can help us with translations on [Crowdin](https://crowdin.com/project/lasuite-docs).
Your language is not there? Request it on our Crowdin page 😊.
## Creating an Issue
@@ -28,7 +37,7 @@ All commit messages must adhere to the following format:
* <**gitmoji**>: Use a gitmoji to represent the purpose of the commit. For example, ✨ for adding a new feature or 🔥 for removing something, see the list here: <https://gitmoji.dev/>.
* **(type)**: Describe the type of change. Common types include `backend`, `frontend`, `CI`, `docker` etc...
* **title**: A short, descriptive title for the change, starting with a lowercase character.
* **title**: A short, descriptive title for the change.
* **description**: Include additional details about what was changed and why.
### Example Commit Message

View File

@@ -51,7 +51,7 @@ COPY ./src/backend /app/
WORKDIR /app
# collectstatic
RUN DJANGO_CONFIGURATION=Build DJANGO_JWT_PRIVATE_SIGNING_KEY=Dummy \
RUN DJANGO_CONFIGURATION=Build \
python manage.py collectstatic --noinput
# Replace duplicated file by a symlink to decrease the overall size of the
@@ -72,10 +72,11 @@ RUN apk add \
gettext \
gdk-pixbuf \
libffi-dev \
pandoc \
pango \
shared-mime-info
RUN wget https://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/conf/mime.types -O /etc/mime.types
# Copy entrypoint
COPY ./docker/files/usr/local/bin/entrypoint /usr/local/bin/entrypoint
@@ -92,6 +93,11 @@ COPY ./src/backend /app/
WORKDIR /app
# Generate compiled translation messages
RUN DJANGO_CONFIGURATION=Build \
python manage.py compilemessages
# We wrap commands run in this container by the following entrypoint that
# creates a user on-the-fly with the container user ID (see USER) and root group
# ID.

View File

@@ -44,7 +44,6 @@ COMPOSE_EXEC_APP = $(COMPOSE_EXEC) app-dev
COMPOSE_RUN = $(COMPOSE) run --rm
COMPOSE_RUN_APP = $(COMPOSE_RUN) app-dev
COMPOSE_RUN_CROWDIN = $(COMPOSE_RUN) crowdin crowdin
WAIT_DB = @$(COMPOSE_RUN) dockerize -wait tcp://$(DB_HOST):$(DB_PORT) -timeout 60s
# -- Backend
MANAGE = $(COMPOSE_RUN_APP) python manage.py
@@ -81,12 +80,12 @@ bootstrap: \
data/static \
create-env-files \
build \
run-with-frontend \
migrate \
demo \
back-i18n-compile \
mails-install \
mails-build
mails-build \
run
.PHONY: bootstrap
# -- Docker/compose
@@ -94,7 +93,6 @@ build: cache ?= --no-cache
build: ## build the project containers
@$(MAKE) build-backend cache=$(cache)
@$(MAKE) build-yjs-provider cache=$(cache)
@$(MAKE) build-frontend cache=$(cache)
.PHONY: build
build-backend: cache ?=
@@ -109,7 +107,7 @@ build-yjs-provider: ## build the y-provider container
build-frontend: cache ?=
build-frontend: ## build the frontend container
@$(COMPOSE) build frontend-dev $(cache)
@$(COMPOSE) build frontend $(cache)
.PHONY: build-frontend
down: ## stop and remove containers, networks, images, and volumes
@@ -120,18 +118,16 @@ logs: ## display app-dev logs (follow mode)
@$(COMPOSE) logs -f app-dev
.PHONY: logs
run: ## start the wsgi (production) and development server
run-backend: ## Start only the backend application and all needed services
@$(COMPOSE) up --force-recreate -d celery-dev
@$(COMPOSE) up --force-recreate -d y-provider
@$(COMPOSE) up --force-recreate -d nginx
@echo "Wait for postgresql to be up..."
@$(WAIT_DB)
.PHONY: run
.PHONY: run-backend
run-with-frontend: ## Start all the containers needed (backend to frontend)
@$(MAKE) run
@$(COMPOSE) up --force-recreate -d frontend-dev
.PHONY: run-with-frontend
run: ## start the wsgi (production) and development server
run:
@$(MAKE) run-backend
.PHONY: run
status: ## an alias for "docker compose ps"
@$(COMPOSE) ps
@@ -188,14 +184,12 @@ test-back-parallel: ## run all back-end tests in parallel
makemigrations: ## run django makemigrations for the impress project.
@echo "$(BOLD)Running makemigrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(WAIT_DB)
@$(MANAGE) makemigrations
.PHONY: makemigrations
migrate: ## run django migrations for the impress project.
@echo "$(BOLD)Running migrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(WAIT_DB)
@$(MANAGE) migrate
.PHONY: migrate
@@ -310,16 +304,16 @@ help:
.PHONY: help
# Front
frontend-install: ## install the frontend locally
frontend-development-install: ## install the frontend locally
cd $(PATH_FRONT_IMPRESS) && yarn
.PHONY: frontend-install
.PHONY: frontend-development-install
frontend-lint: ## run the frontend linter
cd $(PATH_FRONT) && yarn lint
.PHONY: frontend-lint
run-frontend-development: ## Run the frontend in development mode
@$(COMPOSE) stop frontend-dev
@$(COMPOSE) stop frontend
cd $(PATH_FRONT_IMPRESS) && yarn dev
.PHONY: run-frontend-development

195
README.md
View File

@@ -1,113 +1,200 @@
# Impress
<p align="center">
<a href="https://github.com/suitenumerique/docs">
<img alt="Docs" src="/docs/assets/docs-logo.png" width="300" />
</a>
</p>
Impress is a web application for real-time collaborative text editing with user and role based access rights.
Features include :
- User authentication through OIDC
- BlocNote.js text editing experience (markdown support, dynamic conversion, block structure, slash commands for block creation)
- Document export to pdf and docx from predefined templates
- Granular document permissions
- Public link sharing
- Offline mode
<p align="center">
Welcome to Docs! The open source document editor where your notes can become knowledge through live collaboration
</p>
Impress is built on top of [Django Rest Framework](https://www.django-rest-framework.org/), [Next.js](https://nextjs.org/) and [BlocNote.js](https://www.blocknotejs.org/)
<p align="center">
<a href="https://matrix.to/#/#docs-official:matrix.org">
Chat on Matrix
</a> - <a href="/docs/">
Documentation
</a> - <a href="#getting-started-">
Getting started
</a> - <a href="mailto:docs@numerique.gouv.fr">
Reach out
</a>
</p>
## Getting started
<img src="/docs/assets/docs_live_collaboration_light.gif" width="100%" align="center"/>
### Prerequisite
## Why use Docs ❓
Make sure you have a recent version of Docker and [Docker
Compose](https://docs.docker.com/compose/install) installed on your laptop:
Docs is a collaborative text editor designed to address common challenges in knowledge building and sharing.
```bash
$ docker -v
Docker version 20.10.2, build 2291f61
### Write
* 😌 Simple collaborative editing without the formatting complexity of markdown
* 🔌 Offline? No problem, keep writing, your edits will get synced when back online
* 💅 Create clean documents with limited but beautiful formatting options and focus on content
* 🧱 Built for productivity (markdown support, many block types, slash commands, keyboard shortcuts).
* ✨ Save time thanks to our AI actions (generate, sum up, correct, translate)
$ docker compose -v
docker compose version 1.27.4, build 40524192
### Collaborate
* 🤝 Collaborate with your team in real time
* 🔒 Granular access control to ensure your information is secure and only shared with the right people
* 📑 Professional document exports in multiple formats (.odt, .doc, .pdf) with customizable templates
* 📚 Built-in wiki functionality to turn your team's collaborative work into organized knowledge `ETA 02/2025`
### Self-host
* 🚀 Easy to install, scalable and secure alternative to Notion, Outline or Confluence
## Getting started 🔧
### Test it
Test Docs on your browser by logging in on this [environment](https://impress-preprod.beta.numerique.gouv.fr/)
```
email: test.docs@yopmail.com
password: I'd<3ToTestDocs
```
> ⚠️ You may need to run the following commands with `sudo` but this can be
> avoided by assigning your user to the `docker` group.
### Run it locally
### Project bootstrap
> ⚠️ Running Docs locally using the methods described below is for testing purposes only. It is based on building Docs using Minio as the S3 storage solution but you can choose any S3 compatible object storage of your choice.
The easiest way to start working on the project is to use GNU Make:
**Prerequisite**
```bash
Make sure you have a recent version of Docker and [Docker Compose](https://docs.docker.com/compose/install) installed on your laptop:
```shellscript
$ docker -v
Docker version 20.10.2, build 2291f61
$ docker compose version
Docker Compose version v2.32.4
```
> ⚠️ You may need to run the following commands with sudo but this can be avoided by adding your user to the `docker` group.
**Project bootstrap**
The easiest way to start working on the project is to use [GNU Make](https://www.gnu.org/software/make/):
```shellscript
$ make bootstrap FLUSH_ARGS='--no-input'
```
This command builds the `app` container, installs dependencies, performs
database migrations and compile translations. It's a good idea to use this
command each time you are pulling code from the project repository to avoid
dependency-releated or migration-releated issues.
This command builds the `app` container, installs dependencies, performs database migrations and compile translations. It's a good idea to use this command each time you are pulling code from the project repository to avoid dependency-related or migration-related issues.
Your Docker services should now be up and running 🎉
You can access to the project by going to http://localhost:3000.
You can access to the project by going to <http://localhost:3000>.
You will be prompted to log in, the default credentials are:
```bash
```
username: impress
password: impress
```
📝 Note that if you need to run them afterwards, you can use the eponym Make rule:
```bash
$ make run-with-frontend
```shellscript
$ make run
```
---
⚠️ For the frontend developer, it is often better to run the frontend in development mode locally.
⚠️ For the frontend developper, it is often better to run the frontend in development mode locally.
To do so, install the frontend dependencies with the following command:
```bash
$ make frontend-install
```shellscript
$ make frontend-development-install
```
And run the frontend locally in development mode with the following command:
```bash
```shellscript
$ make run-frontend-development
```
To start all the services, except the frontend container, you can use the following command:
```bash
$ make run
```shellscript
$ make run-backend
```
---
### Adding content
**Adding content**
You can create a basic demo site by running:
$ make demo
```shellscript
$ make demo
```
Finally, you can check all available Make rules using:
```bash
```shellscript
$ make help
```
### Django admin
**Django admin**
You can access the Django admin site at
[http://localhost:8071/admin](http://localhost:8071/admin).
<http://localhost:8071/admin>.
You first need to create a superuser account:
```bash
```shellscript
$ make superuser
```
## Contributing
## Feedback 🙋‍♂️🙋‍♀️
This project is intended to be community-driven, so please, do not hesitate to
get in touch if you have any question related to our implementation or design
decisions.
We'd love to hear your thoughts and hear about your experiments, so come and say hi on [Matrix](https://matrix.to/#/#docs-official:matrix.org).
## License
## Roadmap
This work is released under the MIT License (see [LICENSE](./LICENSE)).
Want to know where the project is headed? [🗺️ Checkout our roadmap](https://github.com/orgs/numerique-gouv/projects/13/views/11)
## Licence 📝
This work is released under the MIT License (see [LICENSE](https://github.com/suitenumerique/docs/blob/main/LICENSE)).
While Docs is a public driven initiative our licence choice is an invitation for private sector actors to use, sell and contribute to the project.
## Contributing 🙌
This project is intended to be community-driven, so please, do not hesitate to [get in touch](https://matrix.to/#/#docs-official:matrix.org) if you have any question related to our implementation or design decisions.
You can help us with translations on [Crowdin](https://crowdin.com/project/lasuite-docs).
If you intend to make pull requests see [CONTRIBUTING](https://github.com/suitenumerique/docs/blob/main/CONTRIBUTING.md) for guidelines.
Directory structure:
```markdown
docs
├── bin - executable scripts or binaries that are used for various tasks, such as setup scripts, utility scripts, or custom commands.
├── crowdin - for crowdin translations, a tool or service that helps manage translations for the project.
├── docker - Dockerfiles and related configuration files used to build Docker images for the project. These images can be used for development, testing, or production environments.
├── docs - documentation for the project, including user guides, API documentation, and other helpful resources.
├── env.d/development - environment-specific configuration files for the development environment. These files might include environment variables, configuration settings, or other setup files needed for development.
├── gitlint - configuration files for `gitlint`, a tool that enforces commit message guidelines to ensure consistency and quality in commit messages.
├── playground - experimental or temporary code, where developers can test new features or ideas without affecting the main codebase.
└── src - main source code directory, containing the core application code, libraries, and modules of the project.
```
## Credits ❤️
### Stack
Docs is built on top of [Django Rest Framework](https://www.django-rest-framework.org/), [Next.js](https://nextjs.org/), [BlockNote.js](https://www.blocknotejs.org/), [HocusPocus](https://tiptap.dev/docs/hocuspocus/introduction) and [Yjs](https://yjs.dev/).
### Gov ❤️ open source
Docs is the result of a joint effort led by the French 🇫🇷🥖 ([DINUM](https://www.numerique.gouv.fr/dinum/)) and German 🇩🇪🥨 governments ([ZenDiS](https://zendis.de/)).
We are proud sponsors of [BlockNotejs](https://www.blocknotejs.org/) and [Yjs](https://yjs.dev/).
We are always looking for new public partners (we are currently onboarding the Netherlands 🇳🇱🧀), feel free to [reach out](mailto:docs@numerique.gouv.fr) if you are interested in using or contributing to Docs.
<p align="center">
<img src="/docs/assets/europe_opensource.png" width="50%"/>
</p>

23
SECURITY.md Normal file
View File

@@ -0,0 +1,23 @@
# Security Policy
## Reporting a Vulnerability
Security is very important to us.
If you have any issue regarding security, please disclose the information responsibly submiting [this form](https://vdp.numerique.gouv.fr/p/Send-a-report?lang=en) and not by creating an issue on the repository. You can also email us at docs@numerique.gouv.fr
We appreciate your effort to make Docs more secure.
## Vulnerability disclosure policy
Working with security issues in an open source project can be challenging, as we are required to disclose potential problems that could be exploited by attackers. With this in mind, our security fix policy is as follows:
1. The Maintainers team will handle the fix as usual (Pull Request,
release).
2. In the release notes, we will include the identification numbers from the
GitHub Advisory Database (GHSA) and, if applicable, the Common Vulnerabilities
and Exposures (CVE) identifier for the vulnerability.
3. Once this grace period has passed, we will publish the vulnerability.
By adhering to this security policy, we aim to address security concerns
effectively and responsibly in our open source software project.

View File

@@ -15,3 +15,8 @@ the following command inside your docker container:
(Note : in your development environment, you can `make migrate`.)
## [Unreleased]
- AI features are now limited to users who are authenticated. Before this release, even anonymous
users who gained editor access on a document with link reach used to get AI feature.
IF you want anonymous users to keep access on AI features, you must now define the
`AI_ALLOW_REACH_FROM` setting to "public".

View File

@@ -7,7 +7,6 @@ UNSET_USER=0
TERRAFORM_DIRECTORY="./env.d/terraform"
COMPOSE_FILE="${REPO_DIR}/docker-compose.yml"
COMPOSE_PROJECT="docs"
# _set_user: set (or unset) default user id used to run docker commands
@@ -40,9 +39,8 @@ function _set_user() {
# ARGS : docker compose command arguments
function _docker_compose() {
echo "🐳(compose) project: '${COMPOSE_PROJECT}' file: '${COMPOSE_FILE}'"
echo "🐳(compose) file: '${COMPOSE_FILE}'"
docker compose \
-p "${COMPOSE_PROJECT}" \
-f "${COMPOSE_FILE}" \
--project-directory "${REPO_DIR}" \
"$@"

View File

@@ -1,7 +1,7 @@
#
# Your crowdin's credentials
#
api_token_env: CROWDIN_API_TOKEN
api_token_env: CROWDIN_PERSONAL_TOKEN
project_id_env: CROWDIN_PROJECT_ID
base_path_env: CROWDIN_BASE_PATH
@@ -15,11 +15,11 @@ preserve_hierarchy: true
# Files configuration
#
files: [
{
source : "/backend/locale/django.pot",
dest: "/backend-impress.pot",
translation : "/backend/locale/%locale_with_underscore%/LC_MESSAGES/django.po"
},
{
source : "/backend/locale/django.pot",
dest: "/backend-impress.pot",
translation : "/backend/locale/%locale_with_underscore%/LC_MESSAGES/django.po"
},
{
source: "/frontend/packages/i18n/locales/impress/translations-crowdin.json",
dest: "/frontend-impress.json",

View File

@@ -1,6 +1,13 @@
name: docs
services:
postgresql:
image: postgres:16
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 1s
timeout: 2s
retries: 300
env_file:
- env.d/development/postgresql
ports:
@@ -23,6 +30,11 @@ services:
ports:
- '9000:9000'
- '9001:9001'
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 1s
timeout: 20s
retries: 300
entrypoint: ""
command: minio server --console-address :9001 /data
volumes:
@@ -31,7 +43,9 @@ services:
createbuckets:
image: minio/mc
depends_on:
- minio
minio:
condition: service_healthy
restart: true
entrypoint: >
sh -c "
/usr/bin/mc alias set impress http://minio:9000 impress password && \
@@ -59,10 +73,15 @@ services:
- ./src/backend:/app
- ./data/static:/data/static
depends_on:
- postgresql
- mailcatcher
- redis
- createbuckets
postgresql:
condition: service_healthy
restart: true
mailcatcher:
condition: service_started
redis:
condition: service_started
createbuckets:
condition: service_started
celery-dev:
user: ${DOCKER_USER:-1000}
@@ -93,9 +112,13 @@ services:
- env.d/development/common
- env.d/development/postgresql
depends_on:
- postgresql
- redis
- minio
postgresql:
condition: service_healthy
restart: true
redis:
condition: service_started
minio:
condition: service_started
celery:
user: ${DOCKER_USER:-1000}
@@ -116,11 +139,15 @@ services:
volumes:
- ./docker/files/etc/nginx/conf.d:/etc/nginx/conf.d:ro
depends_on:
- keycloak
- app-dev
- y-provider
app-dev:
condition: service_started
y-provider:
condition: service_started
keycloak:
condition: service_healthy
restart: true
frontend-dev:
frontend:
user: "${DOCKER_USER:-1000}"
build:
context: .
@@ -135,9 +162,6 @@ services:
ports:
- "3000:3000"
dockerize:
image: jwilder/dockerize
crowdin:
image: crowdin/cli:3.16.0
volumes:
@@ -151,7 +175,7 @@ services:
image: node:18
user: "${DOCKER_USER:-1000}"
environment:
HOME: /tmp
HOME: /tmp
volumes:
- ".:/app"
@@ -169,6 +193,11 @@ services:
kc_postgresql:
image: postgres:14.3
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 1s
timeout: 2s
retries: 300
ports:
- "5433:5432"
env_file:
@@ -187,6 +216,13 @@ services:
- --hostname-admin-url=http://localhost:8083/
- --hostname-strict=false
- --hostname-strict-https=false
- --health-enabled=true
- --metrics-enabled=true
healthcheck:
test: ["CMD", "curl", "--head", "-fsS", "http://localhost:8080/health/ready"]
interval: 1s
timeout: 2s
retries: 300
environment:
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: admin
@@ -200,4 +236,6 @@ services:
ports:
- "8080:8080"
depends_on:
- kc_postgresql
kc_postgresql:
condition: service_healthy
restart: true

View File

@@ -68,6 +68,8 @@ server {
# Get resource from Minio
proxy_pass http://minio:9000/impress-media-storage/;
proxy_set_header Host minio:9000;
add_header Content-Security-Policy "default-src 'none'" always;
}
location /media-auth {
@@ -88,5 +90,11 @@ server {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Increase proxy buffer size to allow keycloak to send large
# header responses when a user is created.
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
}
}

BIN
docs/assets/docs-logo.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.8 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 80 KiB

BIN
docs/assets/logo.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.3 KiB

View File

@@ -40,6 +40,7 @@ backend:
LOGIN_REDIRECT_URL: https://impress.127.0.0.1.nip.io
LOGIN_REDIRECT_URL_FAILURE: https://impress.127.0.0.1.nip.io
LOGOUT_REDIRECT_URL: https://impress.127.0.0.1.nip.io
POSTHOG_KEY: "{'id': 'posthog_key', 'host': 'https://product.impress.127.0.0.1.nip.io'}"
DB_HOST: postgresql
DB_NAME: impress
DB_USER: dinum
@@ -121,6 +122,12 @@ yProvider:
COLLABORATION_SERVER_SECRET: my-secret
Y_PROVIDER_API_KEY: my-secret
posthog:
ingress:
enabled: false
ingressAssets:
enabled: false
ingress:
enabled: true
host: impress.127.0.0.1.nip.io

View File

@@ -61,4 +61,4 @@ COLLABORATION_SERVER_SECRET=my-secret
COLLABORATION_WS_URL=ws://localhost:8083/collaboration/ws/
# Frontend
FRONTEND_THEME=dsfr
FRONTEND_THEME=default

View File

@@ -1,3 +1,3 @@
CROWDIN_API_TOKEN=Your-Api-Token
CROWDIN_PERSONAL_TOKEN=Your-Personal-Token
CROWDIN_PROJECT_ID=Your-Project-Id
CROWDIN_BASE_PATH=/app/src

View File

@@ -31,7 +31,7 @@ class GitmojiTitle(LineRule):
"https://raw.githubusercontent.com/carloscuesta/gitmoji/master/packages/gitmojis/src/gitmojis.json"
).json()["gitmojis"]
emojis = [item["emoji"] for item in gitmojis]
pattern = r"^({:s})\(.*\)\s[a-z].*$".format("|".join(emojis))
pattern = r"^({:s})\(.*\)\s[a-zA-Z].*$".format("|".join(emojis))
if not re.search(pattern, title):
violation_msg = 'Title does not match regex "<gitmoji>(<scope>) <subject>"'
return [RuleViolation(self.id, violation_msg, title)]

27
publiccode.yml Normal file
View File

@@ -0,0 +1,27 @@
publiccodeYmlVersion: "2.4.0"
name: Docs
url: https://github.com/suitenumerique/docs
landingURL: https://github.com/suitenumerique/docs
creationDate: 2023-12-10
logo: https://raw.githubusercontent.com/suitenumerique/docs/main/docs/assets/docs-logo.png
usedBy:
- Direction interministériel du numérique (DINUM)
fundedBy:
- name: Direction interministériel du numérique (DINUM)
url: https://www.numerique.gouv.fr
roadmap: "https://github.com/orgs/suitenumerique/projects/2/views/1"
softwareType: "standalone/other"
description:
en:
shortDescription: "The open source document editor where your notes can become knowledge through live collaboration"
fr:
shortDescription: "L'éditeur de documents open source où vos notes peuvent devenir des connaissances grâce à la collaboration en direct."
legal:
license: MIT
maintenance:
type: internal
contacts:
- name: "Virgile Deville"
email: "virgile.deville@numerique.gouv.fr"
- name: "samuel.paccoud"
email: "samuel.paccoud@numerique.gouv.fr"

View File

@@ -14,10 +14,10 @@
"groupName": "ignored js dependencies",
"matchManagers": ["npm"],
"matchPackageNames": [
"eslint",
"fetch-mock",
"node",
"node-fetch",
"eslint",
"workbox-webpack-plugin"
]
}

View File

@@ -4,12 +4,16 @@ from django.contrib import admin
from django.contrib.auth import admin as auth_admin
from django.utils.translation import gettext_lazy as _
from treebeard.admin import TreeAdmin
from treebeard.forms import movenodeform_factory
from . import models
class TemplateAccessInline(admin.TabularInline):
"""Inline admin class for template accesses."""
autocomplete_fields = ["user"]
model = models.TemplateAccess
extra = 0
@@ -111,14 +115,47 @@ class TemplateAdmin(admin.ModelAdmin):
class DocumentAccessInline(admin.TabularInline):
"""Inline admin class for template accesses."""
autocomplete_fields = ["user"]
model = models.DocumentAccess
extra = 0
@admin.register(models.Document)
class DocumentAdmin(admin.ModelAdmin):
class DocumentAdmin(TreeAdmin):
"""Document admin interface declaration."""
fieldsets = (
(
None,
{
"fields": (
"id",
"title",
)
},
),
(
_("Permissions"),
{
"fields": (
"creator",
"link_reach",
"link_role",
)
},
),
(
_("Tree structure"),
{
"fields": (
"path",
"depth",
"numchild",
)
},
),
)
form = movenodeform_factory(models.Document)
inlines = (DocumentAccessInline,)
list_display = (
"id",
@@ -128,6 +165,14 @@ class DocumentAdmin(admin.ModelAdmin):
"created_at",
"updated_at",
)
readonly_fields = (
"creator",
"depth",
"id",
"numchild",
"path",
)
search_fields = ("id", "title")
@admin.register(models.Invitation)

View File

@@ -17,9 +17,10 @@ def exception_handler(exc, context):
https://gist.github.com/twidi/9d55486c36b6a51bdcb05ce3a763e79f
"""
if isinstance(exc, ValidationError):
detail = exc.message_dict
if hasattr(exc, "message"):
detail = None
if hasattr(exc, "message_dict"):
detail = exc.message_dict
elif hasattr(exc, "message"):
detail = exc.message
elif hasattr(exc, "messages"):
detail = exc.messages

View File

@@ -12,19 +12,30 @@ class DocumentFilter(django_filters.FilterSet):
Custom filter for filtering documents.
"""
is_creator_me = django_filters.BooleanFilter(
method="filter_is_creator_me", label=_("Creator is me")
)
is_favorite = django_filters.BooleanFilter(
method="filter_is_favorite", label=_("Favorite")
)
title = django_filters.CharFilter(
field_name="title", lookup_expr="icontains", label=_("Title")
)
class Meta:
model = models.Document
fields = ["is_creator_me", "is_favorite", "link_reach", "title"]
fields = ["title"]
class ListDocumentFilter(DocumentFilter):
"""
Custom filter for filtering documents.
"""
is_creator_me = django_filters.BooleanFilter(
method="filter_is_creator_me", label=_("Creator is me")
)
is_favorite = django_filters.BooleanFilter(
method="filter_is_favorite", label=_("Favorite")
)
class Meta:
model = models.Document
fields = ["is_creator_me", "is_favorite", "title"]
# pylint: disable=unused-argument
def filter_is_creator_me(self, queryset, name, value):
@@ -63,7 +74,4 @@ class DocumentFilter(django_filters.FilterSet):
if not user.is_authenticated:
return queryset
if value:
return queryset.filter(favorited_by_users__user=user)
return queryset.exclude(favorited_by_users__user=user)
return queryset.filter(is_favorite=bool(value))

View File

@@ -2,13 +2,15 @@
from django.core import exceptions
from django.db.models import Q
from django.http import Http404
from rest_framework import permissions
from core.models import DocumentAccess, RoleChoices
from core.models import DocumentAccess, RoleChoices, get_trashbin_cutoff
ACTION_FOR_METHOD_TO_PERMISSION = {
"versions_detail": {"DELETE": "versions_destroy", "GET": "versions_retrieve"}
"versions_detail": {"DELETE": "versions_destroy", "GET": "versions_retrieve"},
"children": {"GET": "children_list", "POST": "children_create"},
}
@@ -109,3 +111,26 @@ class AccessPermission(permissions.BasePermission):
except KeyError:
pass
return abilities.get(action, False)
class DocumentAccessPermission(AccessPermission):
"""Subclass to handle soft deletion specificities."""
def has_object_permission(self, request, view, obj):
"""
Return a 404 on deleted documents
- for which the trashbin cutoff is past
- for which the current user is not owner of the document or one of its ancestors
"""
if (
deleted_at := obj.ancestors_deleted_at
) and deleted_at < get_trashbin_cutoff():
raise Http404
# Compute permission first to ensure the "user_roles" attribute is set
has_permission = super().has_object_permission(request, view, obj)
if obj.ancestors_deleted_at and not RoleChoices.OWNER in obj.user_roles:
raise Http404
return has_permission

View File

@@ -23,7 +23,7 @@ class UserSerializer(serializers.ModelSerializer):
class Meta:
model = models.User
fields = ["id", "email", "full_name", "short_name"]
fields = ["id", "email", "full_name", "short_name", "language"]
read_only_fields = ["id", "email", "full_name", "short_name"]
@@ -128,53 +128,79 @@ class TemplateAccessSerializer(BaseAccessSerializer):
read_only_fields = ["id", "abilities"]
class BaseResourceSerializer(serializers.ModelSerializer):
"""Serialize documents."""
abilities = serializers.SerializerMethodField(read_only=True)
accesses = TemplateAccessSerializer(many=True, read_only=True)
def get_abilities(self, document) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return document.get_abilities(request.user)
return {}
class ListDocumentSerializer(BaseResourceSerializer):
class ListDocumentSerializer(serializers.ModelSerializer):
"""Serialize documents with limited fields for display in lists."""
is_favorite = serializers.BooleanField(read_only=True)
nb_accesses = serializers.IntegerField(read_only=True)
nb_accesses_ancestors = serializers.IntegerField(read_only=True)
nb_accesses_direct = serializers.IntegerField(read_only=True)
user_roles = serializers.SerializerMethodField(read_only=True)
abilities = serializers.SerializerMethodField(read_only=True)
class Meta:
model = models.Document
fields = [
"id",
"abilities",
"content",
"created_at",
"creator",
"depth",
"excerpt",
"is_favorite",
"link_role",
"link_reach",
"nb_accesses",
"nb_accesses_ancestors",
"nb_accesses_direct",
"numchild",
"path",
"title",
"updated_at",
"user_roles",
]
read_only_fields = [
"id",
"abilities",
"created_at",
"creator",
"depth",
"excerpt",
"is_favorite",
"link_role",
"link_reach",
"nb_accesses",
"nb_accesses_ancestors",
"nb_accesses_direct",
"numchild",
"path",
"updated_at",
"user_roles",
]
def get_abilities(self, document) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
paths_links_mapping = self.context.get("paths_links_mapping", None)
# Retrieve ancestor links from paths_links_mapping (if provided)
ancestors_links = (
paths_links_mapping.get(document.path[: -document.steplen])
if paths_links_mapping
else None
)
return document.get_abilities(request.user, ancestors_links=ancestors_links)
return {}
def get_user_roles(self, document):
"""
Return roles of the logged-in user for the current document,
taking into account ancestors.
"""
request = self.context.get("request")
if request:
return document.get_roles(request.user)
return []
class DocumentSerializer(ListDocumentSerializer):
"""Serialize documents with all fields for display in detail views."""
@@ -189,23 +215,34 @@ class DocumentSerializer(ListDocumentSerializer):
"content",
"created_at",
"creator",
"depth",
"excerpt",
"is_favorite",
"link_role",
"link_reach",
"nb_accesses",
"nb_accesses_ancestors",
"nb_accesses_direct",
"numchild",
"path",
"title",
"updated_at",
"user_roles",
]
read_only_fields = [
"id",
"abilities",
"created_at",
"creator",
"depth",
"is_favorite",
"link_role",
"link_reach",
"nb_accesses",
"nb_accesses_ancestors",
"nb_accesses_direct",
"numchild",
"path",
"updated_at",
"user_roles",
]
def get_fields(self):
@@ -287,7 +324,7 @@ class ServerCreateDocumentSerializer(serializers.Serializer):
{"content": ["Could not convert content"]}
) from err
document = models.Document.objects.create(
document = models.Document.add_root(
title=validated_data["title"],
content=document_content,
creator=user,
@@ -330,7 +367,7 @@ class ServerCreateDocumentSerializer(serializers.Serializer):
raise NotImplementedError("Update is not supported for this serializer.")
class LinkDocumentSerializer(BaseResourceSerializer):
class LinkDocumentSerializer(serializers.ModelSerializer):
"""
Serialize link configuration for documents.
We expose it separately from document in order to simplify and secure access control.
@@ -388,6 +425,8 @@ class FileUploadSerializer(serializers.Serializer):
raise serializers.ValidationError("Could not determine file extension.")
self.context["expected_extension"] = extension
self.context["content_type"] = magic_mime_type
self.context["file_name"] = file.name
return file
@@ -395,12 +434,17 @@ class FileUploadSerializer(serializers.Serializer):
"""Override validate to add the computed extension to validated_data."""
attrs["expected_extension"] = self.context["expected_extension"]
attrs["is_unsafe"] = self.context["is_unsafe"]
attrs["content_type"] = self.context["content_type"]
attrs["file_name"] = self.context["file_name"]
return attrs
class TemplateSerializer(BaseResourceSerializer):
class TemplateSerializer(serializers.ModelSerializer):
"""Serialize templates."""
abilities = serializers.SerializerMethodField(read_only=True)
accesses = TemplateAccessSerializer(many=True, read_only=True)
class Meta:
model = models.Template
fields = [
@@ -414,6 +458,13 @@ class TemplateSerializer(BaseResourceSerializer):
]
read_only_fields = ["id", "accesses", "abilities"]
def get_abilities(self, document) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return document.get_abilities(request.user)
return {}
# pylint: disable=abstract-method
class DocumentGenerationSerializer(serializers.Serializer):
@@ -537,3 +588,37 @@ class AITranslateSerializer(serializers.Serializer):
if len(value.strip()) == 0:
raise serializers.ValidationError("Text field cannot be empty.")
return value
class MoveDocumentSerializer(serializers.Serializer):
"""
Serializer for validating input data to move a document within the tree structure.
Fields:
- target_document_id (UUIDField): The ID of the target parent document where the
document should be moved. This field is required and must be a valid UUID.
- position (ChoiceField): Specifies the position of the document in relation to
the target parent's children.
Choices:
- "first-child": Place the document as the first child of the target parent.
- "last-child": Place the document as the last child of the target parent (default).
- "left": Place the document as the left sibling of the target parent.
- "right": Place the document as the right sibling of the target parent.
Example:
Input payload for moving a document:
{
"target_document_id": "123e4567-e89b-12d3-a456-426614174000",
"position": "first-child"
}
Notes:
- The `target_document_id` is mandatory.
- The `position` defaults to "last-child" if not provided.
"""
target_document_id = serializers.UUIDField(required=True)
position = serializers.ChoiceField(
choices=enums.MoveNodePositionChoices.choices,
default=enums.MoveNodePositionChoices.LAST_CHILD,
)

View File

@@ -11,6 +11,58 @@ import botocore
from rest_framework.throttling import BaseThrottle
def nest_tree(flat_list, steplen):
"""
Convert a flat list of serialized documents into a nested tree making advantage
of the`path` field and its step length.
"""
node_dict = {}
roots = []
# Sort the flat list by path to ensure parent nodes are processed first
flat_list.sort(key=lambda x: x["path"])
for node in flat_list:
node["children"] = [] # Initialize children list
node_dict[node["path"]] = node
# Determine parent path
parent_path = node["path"][:-steplen]
if parent_path in node_dict:
node_dict[parent_path]["children"].append(node)
else:
roots.append(node) # Collect root nodes
if len(roots) > 1:
raise ValueError("More than one root element detected.")
return roots[0] if roots else None
def filter_root_paths(paths, skip_sorting=False):
"""
Filters root paths from a list of paths representing a tree structure.
A root path is defined as a path that is not a prefix of any other path.
Args:
paths (list of str): The list of paths.
Returns:
list of str: The filtered list of root paths.
"""
if not skip_sorting:
paths.sort()
root_paths = []
for path in paths:
# If the current path is not a prefix of the last added root path, add it
if not root_paths or not path.startswith(root_paths[-1]):
root_paths.append(path)
return root_paths
def generate_s3_authorization_headers(key):
"""
Generate authorization headers for an s3 object.

File diff suppressed because it is too large Load Diff

View File

@@ -3,6 +3,7 @@ Core application enums declaration
"""
from django.conf import global_settings
from django.db import models
from django.utils.translation import gettext_lazy as _
# In Django's code base, `LANGUAGES` is set by default with all supported languages.
@@ -10,3 +11,14 @@ from django.utils.translation import gettext_lazy as _
# active in the app.
# pylint: disable=no-member
ALL_LANGUAGES = {language: _(name) for language, name in global_settings.LANGUAGES}
class MoveNodePositionChoices(models.TextChoices):
"""Defines the possible positions when moving a django-treebeard node."""
FIRST_CHILD = "first-child", _("First child")
LAST_CHILD = "last-child", _("Last child")
FIRST_SIBLING = "first-sibling", _("First sibling")
LAST_SIBLING = "last-sibling", _("Last sibling")
LEFT = "left", _("Left")
RIGHT = "right", _("Right")

View File

@@ -46,6 +46,23 @@ class UserFactory(factory.django.DjangoModelFactory):
UserTemplateAccessFactory(user=self, role="owner")
class ParentNodeFactory(factory.declarations.ParameteredAttribute):
"""Custom factory attribute for setting the parent node."""
def generate(self, step, params):
"""
Generate a parent node for the factory.
This method is invoked during the factory's build process to determine the parent
node of the current object being created. If `params` is provided, it uses the factory's
metadata to recursively create or fetch the parent node. Otherwise, it returns `None`.
"""
if not params:
return None
subfactory = step.builder.factory_meta.factory
return step.recurse(subfactory, params)
class DocumentFactory(factory.django.DjangoModelFactory):
"""A factory to create documents"""
@@ -54,9 +71,13 @@ class DocumentFactory(factory.django.DjangoModelFactory):
django_get_or_create = ("title",)
skip_postgeneration_save = True
parent = ParentNodeFactory()
title = factory.Sequence(lambda n: f"document{n}")
excerpt = factory.Sequence(lambda n: f"excerpt{n}")
content = factory.Sequence(lambda n: f"content{n}")
creator = factory.SubFactory(UserFactory)
deleted_at = None
link_reach = factory.fuzzy.FuzzyChoice(
[a[0] for a in models.LinkReachChoices.choices]
)
@@ -64,6 +85,29 @@ class DocumentFactory(factory.django.DjangoModelFactory):
[r[0] for r in models.LinkRoleChoices.choices]
)
@classmethod
def _create(cls, model_class, *args, **kwargs):
"""
Custom creation logic for the factory: creates a document as a child node if
a parent is provided; otherwise, creates it as a root node.
"""
parent = kwargs.pop("parent", None)
if parent:
# Add as a child node
kwargs["ancestors_deleted_at"] = (
kwargs.get("ancestors_deleted_at") or parent.ancestors_deleted_at
)
return parent.add_child(instance=model_class(**kwargs))
# Add as a root node
return model_class.add_root(instance=model_class(**kwargs))
@factory.lazy_attribute
def ancestors_deleted_at(self):
"""Should always be set when "deleted_at" is set."""
return self.deleted_at
@factory.post_generation
def users(self, create, extracted, **kwargs):
"""Add users to document from a given list of users with or without roles."""
@@ -74,6 +118,16 @@ class DocumentFactory(factory.django.DjangoModelFactory):
else:
UserDocumentAccessFactory(document=self, user=item[0], role=item[1])
@factory.post_generation
def teams(self, create, extracted, **kwargs):
"""Add teams to document from a given list of teams with or without roles."""
if create and extracted:
for item in extracted:
if isinstance(item, str):
TeamDocumentAccessFactory(document=self, team=item)
else:
TeamDocumentAccessFactory(document=self, team=item[0], role=item[1])
@factory.post_generation
def link_traces(self, create, extracted, **kwargs):
"""Add link traces to document from a given list of users."""

View File

View File

@@ -0,0 +1,95 @@
"""Management command updating the metadata for all the files in the MinIO bucket."""
from django.core.files.storage import default_storage
from django.core.management.base import BaseCommand
import magic
from core.models import Document
# pylint: disable=too-many-locals, broad-exception-caught
class Command(BaseCommand):
"""Update the metadata for all the files in the MinIO bucket."""
help = __doc__
def handle(self, *args, **options):
"""Execute management command."""
s3_client = default_storage.connection.meta.client
bucket_name = default_storage.bucket_name
mime_detector = magic.Magic(mime=True)
documents = Document.objects.all()
self.stdout.write(
f"[INFO] Found {documents.count()} documents. Starting ContentType fix..."
)
for doc in documents:
doc_id_str = str(doc.id)
prefix = f"{doc_id_str}/attachments/"
self.stdout.write(
f"[INFO] Processing attachments under prefix '{prefix}' ..."
)
continuation_token = None
total_updated = 0
while True:
list_kwargs = {"Bucket": bucket_name, "Prefix": prefix}
if continuation_token:
list_kwargs["ContinuationToken"] = continuation_token
response = s3_client.list_objects_v2(**list_kwargs)
# If no objects found under this prefix, break out of the loop
if "Contents" not in response:
break
for obj in response["Contents"]:
key = obj["Key"]
# Skip if it's a folder
if key.endswith("/"):
continue
try:
# Get existing metadata
head_resp = s3_client.head_object(Bucket=bucket_name, Key=key)
# Read first ~1KB for MIME detection
partial_obj = s3_client.get_object(
Bucket=bucket_name, Key=key, Range="bytes=0-1023"
)
partial_data = partial_obj["Body"].read()
# Detect MIME type
magic_mime_type = mime_detector.from_buffer(partial_data)
# Update ContentType
s3_client.copy_object(
Bucket=bucket_name,
CopySource={"Bucket": bucket_name, "Key": key},
Key=key,
ContentType=magic_mime_type,
Metadata=head_resp.get("Metadata", {}),
MetadataDirective="REPLACE",
)
total_updated += 1
except Exception as exc: # noqa
self.stderr.write(
f"[ERROR] Could not update ContentType for {key}: {exc}"
)
if response.get("IsTruncated"):
continuation_token = response.get("NextContinuationToken")
else:
break
if total_updated > 0:
self.stdout.write(
f"[INFO] -> Updated {total_updated} objects for Document {doc_id_str}."
)

View File

@@ -1,166 +1,552 @@
# Generated by Django 5.0.3 on 2024-05-28 20:29
import uuid
import django.contrib.auth.models
import django.core.validators
import django.db.models.deletion
import timezone_field.fields
import uuid
from django.conf import settings
from django.db import migrations, models
import timezone_field.fields
class Migration(migrations.Migration):
initial = True
dependencies = [
('auth', '0012_alter_user_first_name_max_length'),
("auth", "0012_alter_user_first_name_max_length"),
]
operations = [
migrations.CreateModel(
name='Document',
name="Document",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('title', models.CharField(max_length=255, verbose_name='title')),
('is_public', models.BooleanField(default=False, help_text='Whether this document is public for anyone to use.', verbose_name='public')),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
("title", models.CharField(max_length=255, verbose_name="title")),
(
"is_public",
models.BooleanField(
default=False,
help_text="Whether this document is public for anyone to use.",
verbose_name="public",
),
),
],
options={
'verbose_name': 'Document',
'verbose_name_plural': 'Documents',
'db_table': 'impress_document',
'ordering': ('title',),
"verbose_name": "Document",
"verbose_name_plural": "Documents",
"db_table": "impress_document",
"ordering": ("title",),
},
),
migrations.CreateModel(
name='Template',
name="Template",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('title', models.CharField(max_length=255, verbose_name='title')),
('description', models.TextField(blank=True, verbose_name='description')),
('code', models.TextField(blank=True, verbose_name='code')),
('css', models.TextField(blank=True, verbose_name='css')),
('is_public', models.BooleanField(default=False, help_text='Whether this template is public for anyone to use.', verbose_name='public')),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
("title", models.CharField(max_length=255, verbose_name="title")),
(
"description",
models.TextField(blank=True, verbose_name="description"),
),
("code", models.TextField(blank=True, verbose_name="code")),
("css", models.TextField(blank=True, verbose_name="css")),
(
"is_public",
models.BooleanField(
default=False,
help_text="Whether this template is public for anyone to use.",
verbose_name="public",
),
),
],
options={
'verbose_name': 'Template',
'verbose_name_plural': 'Templates',
'db_table': 'impress_template',
'ordering': ('title',),
"verbose_name": "Template",
"verbose_name_plural": "Templates",
"db_table": "impress_template",
"ordering": ("title",),
},
),
migrations.CreateModel(
name='User',
name="User",
fields=[
('password', models.CharField(max_length=128, verbose_name='password')),
('last_login', models.DateTimeField(blank=True, null=True, verbose_name='last login')),
('is_superuser', models.BooleanField(default=False, help_text='Designates that this user has all permissions without explicitly assigning them.', verbose_name='superuser status')),
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('sub', models.CharField(blank=True, help_text='Required. 255 characters or fewer. Letters, numbers, and @/./+/-/_ characters only.', max_length=255, null=True, unique=True, validators=[django.core.validators.RegexValidator(message='Enter a valid sub. This value may contain only letters, numbers, and @/./+/-/_ characters.', regex='^[\\w.@+-]+\\Z')], verbose_name='sub')),
('email', models.EmailField(blank=True, max_length=254, null=True, verbose_name='identity email address')),
('admin_email', models.EmailField(blank=True, max_length=254, null=True, unique=True, verbose_name='admin email address')),
('language', models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language')),
('timezone', timezone_field.fields.TimeZoneField(choices_display='WITH_GMT_OFFSET', default='UTC', help_text='The timezone in which the user wants to see times.', use_pytz=False)),
('is_device', models.BooleanField(default=False, help_text='Whether the user is a device or a real user.', verbose_name='device')),
('is_staff', models.BooleanField(default=False, help_text='Whether the user can log into this admin site.', verbose_name='staff status')),
('is_active', models.BooleanField(default=True, help_text='Whether this user should be treated as active. Unselect this instead of deleting accounts.', verbose_name='active')),
('groups', models.ManyToManyField(blank=True, help_text='The groups this user belongs to. A user will get all permissions granted to each of their groups.', related_name='user_set', related_query_name='user', to='auth.group', verbose_name='groups')),
('user_permissions', models.ManyToManyField(blank=True, help_text='Specific permissions for this user.', related_name='user_set', related_query_name='user', to='auth.permission', verbose_name='user permissions')),
("password", models.CharField(max_length=128, verbose_name="password")),
(
"last_login",
models.DateTimeField(
blank=True, null=True, verbose_name="last login"
),
),
(
"is_superuser",
models.BooleanField(
default=False,
help_text="Designates that this user has all permissions without explicitly assigning them.",
verbose_name="superuser status",
),
),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
(
"sub",
models.CharField(
blank=True,
help_text="Required. 255 characters or fewer. Letters, numbers, and @/./+/-/_ characters only.",
max_length=255,
null=True,
unique=True,
validators=[
django.core.validators.RegexValidator(
message="Enter a valid sub. This value may contain only letters, numbers, and @/./+/-/_ characters.",
regex="^[\\w.@+-]+\\Z",
)
],
verbose_name="sub",
),
),
(
"email",
models.EmailField(
blank=True,
max_length=254,
null=True,
verbose_name="identity email address",
),
),
(
"admin_email",
models.EmailField(
blank=True,
max_length=254,
null=True,
unique=True,
verbose_name="admin email address",
),
),
(
"language",
models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
(
"timezone",
timezone_field.fields.TimeZoneField(
choices_display="WITH_GMT_OFFSET",
default="UTC",
help_text="The timezone in which the user wants to see times.",
use_pytz=False,
),
),
(
"is_device",
models.BooleanField(
default=False,
help_text="Whether the user is a device or a real user.",
verbose_name="device",
),
),
(
"is_staff",
models.BooleanField(
default=False,
help_text="Whether the user can log into this admin site.",
verbose_name="staff status",
),
),
(
"is_active",
models.BooleanField(
default=True,
help_text="Whether this user should be treated as active. Unselect this instead of deleting accounts.",
verbose_name="active",
),
),
(
"groups",
models.ManyToManyField(
blank=True,
help_text="The groups this user belongs to. A user will get all permissions granted to each of their groups.",
related_name="user_set",
related_query_name="user",
to="auth.group",
verbose_name="groups",
),
),
(
"user_permissions",
models.ManyToManyField(
blank=True,
help_text="Specific permissions for this user.",
related_name="user_set",
related_query_name="user",
to="auth.permission",
verbose_name="user permissions",
),
),
],
options={
'verbose_name': 'user',
'verbose_name_plural': 'users',
'db_table': 'impress_user',
"verbose_name": "user",
"verbose_name_plural": "users",
"db_table": "impress_user",
},
managers=[
('objects', django.contrib.auth.models.UserManager()),
("objects", django.contrib.auth.models.UserManager()),
],
),
migrations.CreateModel(
name='DocumentAccess',
name="DocumentAccess",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('team', models.CharField(blank=True, max_length=100)),
('role', models.CharField(choices=[('reader', 'Reader'), ('editor', 'Editor'), ('administrator', 'Administrator'), ('owner', 'Owner')], default='reader', max_length=20)),
('document', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='accesses', to='core.document')),
('user', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
("team", models.CharField(blank=True, max_length=100)),
(
"role",
models.CharField(
choices=[
("reader", "Reader"),
("editor", "Editor"),
("administrator", "Administrator"),
("owner", "Owner"),
],
default="reader",
max_length=20,
),
),
(
"document",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="accesses",
to="core.document",
),
),
(
"user",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.CASCADE,
to=settings.AUTH_USER_MODEL,
),
),
],
options={
'verbose_name': 'Document/user relation',
'verbose_name_plural': 'Document/user relations',
'db_table': 'impress_document_access',
'ordering': ('-created_at',),
"verbose_name": "Document/user relation",
"verbose_name_plural": "Document/user relations",
"db_table": "impress_document_access",
"ordering": ("-created_at",),
},
),
migrations.CreateModel(
name='Invitation',
name="Invitation",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('email', models.EmailField(max_length=254, verbose_name='email address')),
('role', models.CharField(choices=[('reader', 'Reader'), ('editor', 'Editor'), ('administrator', 'Administrator'), ('owner', 'Owner')], default='reader', max_length=20)),
('document', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='invitations', to='core.document')),
('issuer', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='invitations', to=settings.AUTH_USER_MODEL)),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
(
"email",
models.EmailField(max_length=254, verbose_name="email address"),
),
(
"role",
models.CharField(
choices=[
("reader", "Reader"),
("editor", "Editor"),
("administrator", "Administrator"),
("owner", "Owner"),
],
default="reader",
max_length=20,
),
),
(
"document",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="invitations",
to="core.document",
),
),
(
"issuer",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="invitations",
to=settings.AUTH_USER_MODEL,
),
),
],
options={
'verbose_name': 'Document invitation',
'verbose_name_plural': 'Document invitations',
'db_table': 'impress_invitation',
"verbose_name": "Document invitation",
"verbose_name_plural": "Document invitations",
"db_table": "impress_invitation",
},
),
migrations.CreateModel(
name='TemplateAccess',
name="TemplateAccess",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('team', models.CharField(blank=True, max_length=100)),
('role', models.CharField(choices=[('reader', 'Reader'), ('editor', 'Editor'), ('administrator', 'Administrator'), ('owner', 'Owner')], default='reader', max_length=20)),
('template', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='accesses', to='core.template')),
('user', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
("team", models.CharField(blank=True, max_length=100)),
(
"role",
models.CharField(
choices=[
("reader", "Reader"),
("editor", "Editor"),
("administrator", "Administrator"),
("owner", "Owner"),
],
default="reader",
max_length=20,
),
),
(
"template",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="accesses",
to="core.template",
),
),
(
"user",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.CASCADE,
to=settings.AUTH_USER_MODEL,
),
),
],
options={
'verbose_name': 'Template/user relation',
'verbose_name_plural': 'Template/user relations',
'db_table': 'impress_template_access',
'ordering': ('-created_at',),
"verbose_name": "Template/user relation",
"verbose_name_plural": "Template/user relations",
"db_table": "impress_template_access",
"ordering": ("-created_at",),
},
),
migrations.AddConstraint(
model_name='documentaccess',
constraint=models.UniqueConstraint(condition=models.Q(('user__isnull', False)), fields=('user', 'document'), name='unique_document_user', violation_error_message='This user is already in this document.'),
model_name="documentaccess",
constraint=models.UniqueConstraint(
condition=models.Q(("user__isnull", False)),
fields=("user", "document"),
name="unique_document_user",
violation_error_message="This user is already in this document.",
),
),
migrations.AddConstraint(
model_name='documentaccess',
constraint=models.UniqueConstraint(condition=models.Q(('team__gt', '')), fields=('team', 'document'), name='unique_document_team', violation_error_message='This team is already in this document.'),
model_name="documentaccess",
constraint=models.UniqueConstraint(
condition=models.Q(("team__gt", "")),
fields=("team", "document"),
name="unique_document_team",
violation_error_message="This team is already in this document.",
),
),
migrations.AddConstraint(
model_name='documentaccess',
constraint=models.CheckConstraint(check=models.Q(models.Q(('team', ''), ('user__isnull', False)), models.Q(('team__gt', ''), ('user__isnull', True)), _connector='OR'), name='check_document_access_either_user_or_team', violation_error_message='Either user or team must be set, not both.'),
model_name="documentaccess",
constraint=models.CheckConstraint(
check=models.Q(
models.Q(("team", ""), ("user__isnull", False)),
models.Q(("team__gt", ""), ("user__isnull", True)),
_connector="OR",
),
name="check_document_access_either_user_or_team",
violation_error_message="Either user or team must be set, not both.",
),
),
migrations.AddConstraint(
model_name='invitation',
constraint=models.UniqueConstraint(fields=('email', 'document'), name='email_and_document_unique_together'),
model_name="invitation",
constraint=models.UniqueConstraint(
fields=("email", "document"), name="email_and_document_unique_together"
),
),
migrations.AddConstraint(
model_name='templateaccess',
constraint=models.UniqueConstraint(condition=models.Q(('user__isnull', False)), fields=('user', 'template'), name='unique_template_user', violation_error_message='This user is already in this template.'),
model_name="templateaccess",
constraint=models.UniqueConstraint(
condition=models.Q(("user__isnull", False)),
fields=("user", "template"),
name="unique_template_user",
violation_error_message="This user is already in this template.",
),
),
migrations.AddConstraint(
model_name='templateaccess',
constraint=models.UniqueConstraint(condition=models.Q(('team__gt', '')), fields=('team', 'template'), name='unique_template_team', violation_error_message='This team is already in this template.'),
model_name="templateaccess",
constraint=models.UniqueConstraint(
condition=models.Q(("team__gt", "")),
fields=("team", "template"),
name="unique_template_team",
violation_error_message="This team is already in this template.",
),
),
migrations.AddConstraint(
model_name='templateaccess',
constraint=models.CheckConstraint(check=models.Q(models.Q(('team', ''), ('user__isnull', False)), models.Q(('team__gt', ''), ('user__isnull', True)), _connector='OR'), name='check_template_access_either_user_or_team', violation_error_message='Either user or team must be set, not both.'),
model_name="templateaccess",
constraint=models.CheckConstraint(
check=models.Q(
models.Q(("team", ""), ("user__isnull", False)),
models.Q(("team__gt", ""), ("user__isnull", True)),
_connector="OR",
),
name="check_template_access_either_user_or_team",
violation_error_message="Either user or team must be set, not both.",
),
),
]

View File

@@ -1,9 +1,9 @@
from django.db import migrations
class Migration(migrations.Migration):
class Migration(migrations.Migration):
dependencies = [
('core', '0001_initial'),
("core", "0001_initial"),
]
operations = [

View File

@@ -1,52 +1,114 @@
# Generated by Django 5.1 on 2024-09-08 16:55
import django.db.models.deletion
import uuid
import django.db.models.deletion
from django.conf import settings
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0002_create_pg_trgm_extension'),
("core", "0002_create_pg_trgm_extension"),
]
operations = [
migrations.AddField(
model_name='document',
name='link_reach',
field=models.CharField(choices=[('restricted', 'Restricted'), ('authenticated', 'Authenticated'), ('public', 'Public')], default='authenticated', max_length=20),
model_name="document",
name="link_reach",
field=models.CharField(
choices=[
("restricted", "Restricted"),
("authenticated", "Authenticated"),
("public", "Public"),
],
default="authenticated",
max_length=20,
),
),
migrations.AddField(
model_name='document',
name='link_role',
field=models.CharField(choices=[('reader', 'Reader'), ('editor', 'Editor')], default='reader', max_length=20),
model_name="document",
name="link_role",
field=models.CharField(
choices=[("reader", "Reader"), ("editor", "Editor")],
default="reader",
max_length=20,
),
),
migrations.AlterField(
model_name='document',
name='is_public',
model_name="document",
name="is_public",
field=models.BooleanField(null=True),
),
migrations.AlterField(
model_name='user',
name='language',
field=models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language'),
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
migrations.CreateModel(
name='LinkTrace',
name="LinkTrace",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('document', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='link_traces', to='core.document')),
('user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='link_traces', to=settings.AUTH_USER_MODEL)),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
(
"document",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="link_traces",
to="core.document",
),
),
(
"user",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="link_traces",
to=settings.AUTH_USER_MODEL,
),
),
],
options={
'verbose_name': 'Document/user link trace',
'verbose_name_plural': 'Document/user link traces',
'db_table': 'impress_link_trace',
'constraints': [models.UniqueConstraint(fields=('user', 'document'), name='unique_link_trace_document_user', violation_error_message='A link trace already exists for this document/user.')],
"verbose_name": "Document/user link trace",
"verbose_name_plural": "Document/user link traces",
"db_table": "impress_link_trace",
"constraints": [
models.UniqueConstraint(
fields=("user", "document"),
name="unique_link_trace_document_user",
violation_error_message="A link trace already exists for this document/user.",
)
],
},
),
]

View File

@@ -1,13 +1,14 @@
# Generated by Django 5.1 on 2024-09-08 17:04
from django.db import migrations
def migrate_is_public_to_link_reach(apps, schema_editor):
"""
Forward migration: Migrate 'is_public' to 'link_reach'.
If is_public == True, set link_reach to 'public'
"""
Document = apps.get_model('core', 'Document')
Document.objects.filter(is_public=True).update(link_reach='public')
Document = apps.get_model("core", "Document")
Document.objects.filter(is_public=True).update(link_reach="public")
def reverse_migrate_link_reach_to_is_public(apps, schema_editor):
@@ -16,20 +17,20 @@ def reverse_migrate_link_reach_to_is_public(apps, schema_editor):
- If link_reach == 'public', set is_public to True
- Else set is_public to False
"""
Document = apps.get_model('core', 'Document')
Document.objects.filter(link_reach='public').update(is_public=True)
Document.objects.filter(link_reach__in=['restricted', "authenticated"]).update(is_public=False)
Document = apps.get_model("core", "Document")
Document.objects.filter(link_reach="public").update(is_public=True)
Document.objects.filter(link_reach__in=["restricted", "authenticated"]).update(
is_public=False
)
class Migration(migrations.Migration):
dependencies = [
('core', '0003_document_link_reach_document_link_role_and_more'),
("core", "0003_document_link_reach_document_link_role_and_more"),
]
operations = [
migrations.RunPython(
migrate_is_public_to_link_reach,
reverse_migrate_link_reach_to_is_public
migrate_is_public_to_link_reach, reverse_migrate_link_reach_to_is_public
),
]

View File

@@ -4,15 +4,16 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0004_migrate_is_public_to_link_reach'),
("core", "0004_migrate_is_public_to_link_reach"),
]
operations = [
migrations.AlterField(
model_name='document',
name='title',
field=models.CharField(blank=True, max_length=255, null=True, verbose_name='title'),
model_name="document",
name="title",
field=models.CharField(
blank=True, max_length=255, null=True, verbose_name="title"
),
),
]

View File

@@ -4,25 +4,34 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0005_remove_document_is_public_alter_document_link_reach_and_more'),
("core", "0005_remove_document_is_public_alter_document_link_reach_and_more"),
]
operations = [
migrations.AddField(
model_name='user',
name='full_name',
field=models.CharField(blank=True, max_length=100, null=True, verbose_name='full name'),
model_name="user",
name="full_name",
field=models.CharField(
blank=True, max_length=100, null=True, verbose_name="full name"
),
),
migrations.AddField(
model_name='user',
name='short_name',
field=models.CharField(blank=True, max_length=20, null=True, verbose_name='short name'),
model_name="user",
name="short_name",
field=models.CharField(
blank=True, max_length=20, null=True, verbose_name="short name"
),
),
migrations.AlterField(
model_name='user',
name='language',
field=models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language'),
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
]

View File

@@ -117,10 +117,10 @@ BEGIN
END $$;
"""
class Migration(migrations.Migration):
class Migration(migrations.Migration):
dependencies = [
('core', '0006_add_user_full_name_and_short_name'),
("core", "0006_add_user_full_name_and_short_name"),
]
operations = [

View File

@@ -4,15 +4,22 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0007_fix_users_duplicate'),
("core", "0007_fix_users_duplicate"),
]
operations = [
migrations.AlterField(
model_name='document',
name='link_reach',
field=models.CharField(choices=[('restricted', 'Restricted'), ('authenticated', 'Authenticated'), ('public', 'Public')], default='restricted', max_length=20),
model_name="document",
name="link_reach",
field=models.CharField(
choices=[
("restricted", "Restricted"),
("authenticated", "Authenticated"),
("public", "Public"),
],
default="restricted",
max_length=20,
),
),
]

View File

@@ -1,37 +1,87 @@
# Generated by Django 5.1.2 on 2024-11-08 07:59
import django.db.models.deletion
import uuid
import django.db.models.deletion
from django.conf import settings
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0008_alter_document_link_reach'),
("core", "0008_alter_document_link_reach"),
]
operations = [
migrations.AlterField(
model_name='user',
name='language',
field=models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language'),
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
migrations.CreateModel(
name='DocumentFavorite',
name="DocumentFavorite",
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('document', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='favorited_by_users', to='core.document')),
('user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='favorite_documents', to=settings.AUTH_USER_MODEL)),
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
help_text="primary key for the record as UUID",
primary_key=True,
serialize=False,
verbose_name="id",
),
),
(
"created_at",
models.DateTimeField(
auto_now_add=True,
help_text="date and time at which a record was created",
verbose_name="created on",
),
),
(
"updated_at",
models.DateTimeField(
auto_now=True,
help_text="date and time at which a record was last updated",
verbose_name="updated on",
),
),
(
"document",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="favorited_by_users",
to="core.document",
),
),
(
"user",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="favorite_documents",
to=settings.AUTH_USER_MODEL,
),
),
],
options={
'verbose_name': 'Document favorite',
'verbose_name_plural': 'Document favorites',
'db_table': 'impress_document_favorite',
'constraints': [models.UniqueConstraint(fields=('user', 'document'), name='unique_document_favorite_user', violation_error_message='This document is already targeted by a favorite relation instance for the same user.')],
"verbose_name": "Document favorite",
"verbose_name_plural": "Document favorites",
"db_table": "impress_document_favorite",
"constraints": [
models.UniqueConstraint(
fields=("user", "document"),
name="unique_document_favorite_user",
violation_error_message="This document is already targeted by a favorite relation instance for the same user.",
)
],
},
),
]

View File

@@ -7,25 +7,48 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0009_add_document_favorite'),
("core", "0009_add_document_favorite"),
]
operations = [
migrations.AddField(
model_name='document',
name='creator',
field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.RESTRICT, related_name='documents_created', to=settings.AUTH_USER_MODEL),
model_name="document",
name="creator",
field=models.ForeignKey(
null=True,
on_delete=django.db.models.deletion.RESTRICT,
related_name="documents_created",
to=settings.AUTH_USER_MODEL,
),
),
migrations.AlterField(
model_name='user',
name='language',
field=models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language'),
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
migrations.AlterField(
model_name='user',
name='sub',
field=models.CharField(blank=True, help_text='Required. 255 characters or fewer. Letters, numbers, and @/./+/-/_/: characters only.', max_length=255, null=True, unique=True, validators=[django.core.validators.RegexValidator(message='Enter a valid sub. This value may contain only letters, numbers, and @/./+/-/_/: characters.', regex='^[\\w.@+-:]+\\Z')], verbose_name='sub'),
model_name="user",
name="sub",
field=models.CharField(
blank=True,
help_text="Required. 255 characters or fewer. Letters, numbers, and @/./+/-/_/: characters only.",
max_length=255,
null=True,
unique=True,
validators=[
django.core.validators.RegexValidator(
message="Enter a valid sub. This value may contain only letters, numbers, and @/./+/-/_/: characters.",
regex="^[\\w.@+-:]+\\Z",
)
],
verbose_name="sub",
),
),
]

View File

@@ -3,7 +3,7 @@
import django.db.models.deletion
from django.conf import settings
from django.db import migrations
from django.db.models import F, ForeignKey, Subquery, OuterRef, Q
from django.db.models import F, ForeignKey, OuterRef, Q, Subquery
def set_creator_from_document_access(apps, schema_editor):
@@ -25,28 +25,37 @@ def set_creator_from_document_access(apps, schema_editor):
DocumentAccess = apps.get_model("core", "DocumentAccess")
# Update `creator` using the "owner" role
owner_subquery = DocumentAccess.objects.filter(
document=OuterRef('pk'),
user__isnull=False,
role='owner',
).order_by('created_at').values('user_id')[:1]
owner_subquery = (
DocumentAccess.objects.filter(
document=OuterRef("pk"),
user__isnull=False,
role="owner",
)
.order_by("created_at")
.values("user_id")[:1]
)
Document.objects.filter(
creator__isnull=True
).update(creator=Subquery(owner_subquery))
Document.objects.filter(creator__isnull=True).update(
creator=Subquery(owner_subquery)
)
class Migration(migrations.Migration):
dependencies = [
('core', '0010_add_field_creator_to_document'),
("core", "0010_add_field_creator_to_document"),
]
operations = [
migrations.RunPython(set_creator_from_document_access, reverse_code=migrations.RunPython.noop),
migrations.RunPython(
set_creator_from_document_access, reverse_code=migrations.RunPython.noop
),
migrations.AlterField(
model_name='document',
name='creator',
field=ForeignKey(on_delete=django.db.models.deletion.RESTRICT, related_name='documents_created', to=settings.AUTH_USER_MODEL),
model_name="document",
name="creator",
field=ForeignKey(
on_delete=django.db.models.deletion.RESTRICT,
related_name="documents_created",
to=settings.AUTH_USER_MODEL,
),
),
]

View File

@@ -6,25 +6,42 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0011_populate_creator_field_and_make_it_required'),
("core", "0011_populate_creator_field_and_make_it_required"),
]
operations = [
migrations.AlterField(
model_name='document',
name='creator',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.RESTRICT, related_name='documents_created', to=settings.AUTH_USER_MODEL),
model_name="document",
name="creator",
field=models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.RESTRICT,
related_name="documents_created",
to=settings.AUTH_USER_MODEL,
),
),
migrations.AlterField(
model_name='invitation',
name='issuer',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='invitations', to=settings.AUTH_USER_MODEL),
model_name="invitation",
name="issuer",
field=models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.CASCADE,
related_name="invitations",
to=settings.AUTH_USER_MODEL,
),
),
migrations.AlterField(
model_name='user',
name='language',
field=models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language'),
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
]

View File

@@ -0,0 +1,16 @@
# Generated by Django 5.1.4 on 2025-01-25 08:38
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
("core", "0012_make_document_creator_and_invitation_issuer_optional"),
]
operations = [
migrations.RunSQL(
"CREATE EXTENSION IF NOT EXISTS fuzzystrmatch;",
reverse_sql="DROP EXTENSION IF EXISTS fuzzystrmatch;",
),
]

View File

@@ -0,0 +1,32 @@
# Generated by Django 5.1.2 on 2024-12-07 09:43
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("core", "0013_activate_fuzzystrmatch_extension"),
]
operations = [
migrations.AddField(
model_name="document",
name="depth",
field=models.PositiveIntegerField(default=0),
preserve_default=False,
),
migrations.AddField(
model_name="document",
name="numchild",
field=models.PositiveIntegerField(default=0),
),
migrations.AddField(
model_name="document",
name="path",
# Allow null values pending the next datamigration to populate the field
field=models.CharField(
db_collation="C", max_length=252, null=True, unique=True
),
preserve_default=False,
),
]

View File

@@ -0,0 +1,51 @@
# Generated by Django 5.1.2 on 2024-12-07 10:33
from django.db import migrations, models
from treebeard.numconv import NumConv
ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
STEPLEN = 7
def set_path_on_existing_documents(apps, schema_editor):
"""
Updates the `path` and `depth` fields for all existing Document records
to ensure valid materialized paths.
This function assigns a unique `path` to each Document as a root node
Note: After running this migration, we quickly modify the schema to make
the `path` field required as it should.
"""
Document = apps.get_model("core", "Document")
# Iterate over all existing documents and make them root nodes
documents = Document.objects.order_by("created_at").values_list("id", flat=True)
numconv = NumConv(len(ALPHABET), ALPHABET)
updates = []
for i, pk in enumerate(documents):
key = numconv.int2str(i)
path = "{0}{1}".format(ALPHABET[0] * (STEPLEN - len(key)), key)
updates.append(Document(pk=pk, path=path, depth=1))
# Bulk update using the prepared updates list
Document.objects.bulk_update(updates, ["depth", "path"])
class Migration(migrations.Migration):
dependencies = [
("core", "0014_add_tree_structure_to_documents"),
]
operations = [
migrations.RunPython(
set_path_on_existing_documents, reverse_code=migrations.RunPython.noop
),
migrations.AlterField(
model_name="document",
name="path",
field=models.CharField(db_collation="C", max_length=252, unique=True),
),
]

View File

@@ -0,0 +1,30 @@
# Generated by Django 5.1.4 on 2024-12-18 08:56
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("core", "0015_set_path_on_existing_documents"),
]
operations = [
migrations.AddField(
model_name="document",
name="excerpt",
field=models.TextField(
blank=True, max_length=300, null=True, verbose_name="excerpt"
),
),
migrations.AlterField(
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
]

View File

@@ -0,0 +1,52 @@
# Generated by Django 5.1.4 on 2025-01-12 14:27
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("core", "0016_add_document_excerpt"),
]
operations = [
migrations.AlterModelOptions(
name="document",
options={
"ordering": ("path",),
"verbose_name": "Document",
"verbose_name_plural": "Documents",
},
),
migrations.AddField(
model_name="document",
name="ancestors_deleted_at",
field=models.DateTimeField(blank=True, null=True),
),
migrations.AddField(
model_name="document",
name="deleted_at",
field=models.DateTimeField(blank=True, null=True),
),
migrations.AlterField(
model_name="user",
name="language",
field=models.CharField(
choices="(('en-us', 'English'), ('fr-fr', 'French'), ('de-de', 'German'))",
default="en-us",
help_text="The language in which the user wants to see the interface.",
max_length=10,
verbose_name="language",
),
),
migrations.AddConstraint(
model_name="document",
constraint=models.CheckConstraint(
condition=models.Q(
("deleted_at__isnull", True),
("deleted_at", models.F("ancestors_deleted_at")),
_connector="OR",
),
name="check_deleted_at_matches_ancestors_deleted_at_when_set",
),
),
]

View File

@@ -0,0 +1,24 @@
from django.db import migrations
def update_titles_to_null(apps, schema_editor):
"""
If the titles are "Untitled document" or "Unbenanntes Dokument" or "Document sans titre"
we set them to Null
"""
Document = apps.get_model("core", "Document")
Document.objects.filter(
title__in=["Untitled document", "Unbenanntes Dokument", "Document sans titre"]
).update(title=None)
class Migration(migrations.Migration):
dependencies = [
("core", "0017_add_fields_for_soft_delete"),
]
operations = [
migrations.RunPython(
update_titles_to_null, reverse_code=migrations.RunPython.noop
),
]

View File

@@ -0,0 +1,36 @@
# Generated by Django 5.1.5 on 2025-03-04 12:23
from django.db import migrations, models
import core.models
class Migration(migrations.Migration):
dependencies = [
("core", "0018_update_blank_title"),
]
operations = [
migrations.AlterModelManagers(
name="user",
managers=[
("objects", core.models.UserManager()),
],
),
migrations.AlterField(
model_name="user",
name="language",
field=models.CharField(
blank=True,
choices=[
("en-us", "English"),
("fr-fr", "Français"),
("de-de", "Deutsch"),
],
default=None,
help_text="The language in which the user wants to see the interface.",
max_length=10,
null=True,
verbose_name="language",
),
),
]

View File

@@ -0,0 +1,17 @@
# Generated by Django 5.1.7 on 2025-03-14 14:03
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("core", "0019_alter_user_language_default_to_null"),
]
operations = [
migrations.AddField(
model_name="document",
name="has_deleted_children",
field=models.BooleanField(default=False),
),
]

View File

@@ -5,56 +5,48 @@ Declare and configure the models for the impress core application
import hashlib
import smtplib
import tempfile
import textwrap
import uuid
from collections import defaultdict
from datetime import timedelta
from io import BytesIO
from logging import getLogger
from django.conf import settings
from django.contrib.auth import models as auth_models
from django.contrib.auth.base_user import AbstractBaseUser
from django.contrib.sites.models import Site
from django.core import exceptions, mail, validators
from django.core import mail, validators
from django.core.cache import cache
from django.core.files.base import ContentFile
from django.core.files.storage import default_storage
from django.core.mail import send_mail
from django.db import models
from django.http import FileResponse
from django.template.base import Template as DjangoTemplate
from django.template.context import Context
from django.db import models, transaction
from django.db.models.functions import Left, Length
from django.template.loader import render_to_string
from django.utils import html, timezone
from django.utils import timezone
from django.utils.functional import cached_property, lazy
from django.utils.translation import get_language, override
from django.utils.translation import gettext_lazy as _
import frontmatter
import markdown
import pypandoc
import weasyprint
from botocore.exceptions import ClientError
from rest_framework.exceptions import ValidationError
from timezone_field import TimeZoneField
from treebeard.mp_tree import MP_Node, MP_NodeManager, MP_NodeQuerySet
logger = getLogger(__name__)
def get_resource_roles(resource, user):
"""Compute the roles a user has on a resource."""
if not user.is_authenticated:
return []
def get_trashbin_cutoff():
"""
Calculate the cutoff datetime for soft-deleted items based on the retention policy.
try:
roles = resource.user_roles or []
except AttributeError:
try:
roles = resource.accesses.filter(
models.Q(user=user) | models.Q(team__in=user.teams),
).values_list("role", flat=True)
except (models.ObjectDoesNotExist, IndexError):
roles = []
return roles
The function returns the current datetime minus the number of days specified in
the TRASHBIN_CUTOFF_DAYS setting, indicating the oldest date for items that can
remain in the trash bin.
Returns:
datetime: The cutoff datetime for soft-deleted items.
"""
return timezone.now() - timedelta(days=settings.TRASHBIN_CUTOFF_DAYS)
class LinkRoleChoices(models.TextChoices):
@@ -89,6 +81,55 @@ class LinkReachChoices(models.TextChoices):
) # Any authenticated user can access the document
PUBLIC = "public", _("Public") # Even anonymous users can access the document
@classmethod
def get_select_options(cls, ancestors_links):
"""
Determines the valid select options for link reach and link role depending on the
list of ancestors' link reach/role.
Args:
ancestors_links: List of dictionaries, each with 'link_reach' and 'link_role' keys
representing the reach and role of ancestors links.
Returns:
Dictionary mapping possible reach levels to their corresponding possible roles.
"""
# If no ancestors, return all options
if not ancestors_links:
return dict.fromkeys(cls.values, LinkRoleChoices.values)
# Initialize result with all possible reaches and role options as sets
result = {reach: set(LinkRoleChoices.values) for reach in cls.values}
# Group roles by reach level
reach_roles = defaultdict(set)
for link in ancestors_links:
reach_roles[link["link_reach"]].add(link["link_role"])
# Apply constraints based on ancestor links
if LinkRoleChoices.EDITOR in reach_roles[cls.RESTRICTED]:
result[cls.RESTRICTED].discard(LinkRoleChoices.READER)
if LinkRoleChoices.EDITOR in reach_roles[cls.AUTHENTICATED]:
result[cls.AUTHENTICATED].discard(LinkRoleChoices.READER)
result.pop(cls.RESTRICTED, None)
elif LinkRoleChoices.READER in reach_roles[cls.AUTHENTICATED]:
result[cls.RESTRICTED].discard(LinkRoleChoices.READER)
if LinkRoleChoices.EDITOR in reach_roles[cls.PUBLIC]:
result[cls.PUBLIC].discard(LinkRoleChoices.READER)
result.pop(cls.AUTHENTICATED, None)
result.pop(cls.RESTRICTED, None)
elif LinkRoleChoices.READER in reach_roles[cls.PUBLIC]:
result[cls.AUTHENTICATED].discard(LinkRoleChoices.READER)
result.get(cls.RESTRICTED, set()).discard(LinkRoleChoices.READER)
# Convert roles sets to lists while maintaining the order from LinkRoleChoices
for reach, roles in result.items():
result[reach] = [role for role in LinkRoleChoices.values if role in roles]
return result
class DuplicateEmailError(Exception):
"""Raised when an email is already associated with a pre-existing user."""
@@ -203,9 +244,11 @@ class User(AbstractBaseUser, BaseModel, auth_models.PermissionsMixin):
language = models.CharField(
max_length=10,
choices=lazy(lambda: settings.LANGUAGES, tuple)(),
default=settings.LANGUAGE_CODE,
default=None,
verbose_name=_("language"),
help_text=_("The language in which the user wants to see the interface."),
null=True,
blank=True,
)
timezone = TimeZoneField(
choices_display="WITH_GMT_OFFSET",
@@ -376,10 +419,56 @@ class BaseAccess(BaseModel):
}
class Document(BaseModel):
class DocumentQuerySet(MP_NodeQuerySet):
"""
Custom queryset for the Document model, providing additional methods
to filter documents based on user permissions.
"""
def readable_per_se(self, user):
"""
Filters the queryset to return documents that the given user has
permission to read.
:param user: The user for whom readable documents are to be fetched.
:return: A queryset of documents readable by the user.
"""
if user.is_authenticated:
return self.filter(
models.Q(accesses__user=user)
| models.Q(accesses__team__in=user.teams)
| ~models.Q(link_reach=LinkReachChoices.RESTRICTED)
)
return self.filter(link_reach=LinkReachChoices.PUBLIC)
class DocumentManager(MP_NodeManager):
"""
Custom manager for the Document model, enabling the use of the custom
queryset methods directly from the model manager.
"""
def get_queryset(self):
"""
Overrides the default get_queryset method to return a custom queryset.
:return: An instance of DocumentQuerySet.
"""
return DocumentQuerySet(self.model, using=self._db)
def readable_per_se(self, user):
"""
Filters documents based on user permissions using the custom queryset.
:param user: The user for whom readable documents are to be fetched.
:return: A queryset of documents readable by the user.
"""
return self.get_queryset().readable_per_se(user)
class Document(MP_Node, BaseModel):
"""Pad document carrying the content."""
title = models.CharField(_("title"), max_length=255, null=True, blank=True)
excerpt = models.TextField(_("excerpt"), max_length=300, null=True, blank=True)
link_reach = models.CharField(
max_length=20,
choices=LinkReachChoices.choices,
@@ -395,14 +484,35 @@ class Document(BaseModel):
blank=True,
null=True,
)
deleted_at = models.DateTimeField(null=True, blank=True)
ancestors_deleted_at = models.DateTimeField(null=True, blank=True)
has_deleted_children = models.BooleanField(default=False)
_content = None
# Tree structure
alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
steplen = 7 # nb siblings max: 3,521,614,606,208
node_order_by = [] # Manual ordering
path = models.CharField(max_length=7 * 36, unique=True, db_collation="C")
objects = DocumentManager()
class Meta:
db_table = "impress_document"
ordering = ("title",)
ordering = ("path",)
verbose_name = _("Document")
verbose_name_plural = _("Documents")
constraints = [
models.CheckConstraint(
check=(
models.Q(deleted_at__isnull=True)
| models.Q(deleted_at=models.F("ancestors_deleted_at"))
),
name="check_deleted_at_matches_ancestors_deleted_at_when_set",
),
]
def __str__(self):
return str(self.title) if self.title else str(_("Untitled Document"))
@@ -437,6 +547,12 @@ class Document(BaseModel):
content_file = ContentFile(bytes_content)
default_storage.save(file_key, content_file)
def is_leaf(self):
"""
:returns: True if the node is has no children
"""
return not self.has_deleted_children and self.numchild == 0
@property
def key_base(self):
"""Key base of the location where the document is stored in object storage."""
@@ -473,9 +589,13 @@ class Document(BaseModel):
def get_content_response(self, version_id=""):
"""Get the content in a specific version of the document"""
return default_storage.connection.meta.client.get_object(
Bucket=default_storage.bucket_name, Key=self.file_key, VersionId=version_id
)
params = {
"Bucket": default_storage.bucket_name,
"Key": self.file_key,
}
if version_id:
params["VersionId"] = version_id
return default_storage.connection.meta.client.get_object(**params)
def get_versions_slice(self, from_version_id="", min_datetime=None, page_size=None):
"""Get document versions from object storage with pagination and starting conditions"""
@@ -541,48 +661,166 @@ class Document(BaseModel):
Bucket=default_storage.bucket_name, Key=self.file_key, VersionId=version_id
)
def get_abilities(self, user):
def get_nb_accesses_cache_key(self):
"""Generate a unique cache key for each document."""
return f"document_{self.id!s}_nb_accesses"
def get_nb_accesses(self):
"""
Calculate the number of accesses:
- directly attached to the document
- attached to any of the document's ancestors
"""
cache_key = self.get_nb_accesses_cache_key()
nb_accesses = cache.get(cache_key)
if nb_accesses is None:
nb_accesses = (
DocumentAccess.objects.filter(document=self).count(),
DocumentAccess.objects.filter(
document__path=Left(
models.Value(self.path), Length("document__path")
),
document__ancestors_deleted_at__isnull=True,
).count(),
)
cache.set(cache_key, nb_accesses)
return nb_accesses
@property
def nb_accesses_direct(self):
"""Returns the number of accesses related to the document or one of its ancestors."""
return self.get_nb_accesses()[0]
@property
def nb_accesses_ancestors(self):
"""Returns the number of accesses related to the document or one of its ancestors."""
return self.get_nb_accesses()[1]
def invalidate_nb_accesses_cache(self):
"""
Invalidate the cache for number of accesses, including on affected descendants.
Args:
path: can optionally be passed as argument (useful when invalidating cache for a
document we just deleted)
"""
for document in Document.objects.filter(path__startswith=self.path).only("id"):
cache_key = document.get_nb_accesses_cache_key()
cache.delete(cache_key)
def get_roles(self, user):
"""Return the roles a user has on a document."""
if not user.is_authenticated:
return []
try:
roles = self.user_roles or []
except AttributeError:
try:
roles = DocumentAccess.objects.filter(
models.Q(user=user) | models.Q(team__in=user.teams),
document__path=Left(
models.Value(self.path), Length("document__path")
),
).values_list("role", flat=True)
except (models.ObjectDoesNotExist, IndexError):
roles = []
return roles
def get_links_definitions(self, ancestors_links):
"""Get links reach/role definitions for the current document and its ancestors."""
links_definitions = defaultdict(set)
links_definitions[self.link_reach].add(self.link_role)
# Merge ancestor link definitions
for ancestor in ancestors_links:
links_definitions[ancestor["link_reach"]].add(ancestor["link_role"])
return dict(links_definitions) # Convert defaultdict back to a normal dict
def get_abilities(self, user, ancestors_links=None):
"""
Compute and return abilities for a given user on the document.
"""
roles = set(get_resource_roles(self, user))
if self.depth <= 1 or getattr(self, "is_highest_ancestor_for_user", False):
ancestors_links = []
elif ancestors_links is None:
ancestors_links = self.get_ancestors().values("link_reach", "link_role")
# Compute version roles before adding link roles because we don't
roles = set(
self.get_roles(user)
) # at this point only roles based on specific access
# Characteristics that are based only on specific access
is_owner = RoleChoices.OWNER in roles
is_deleted = self.ancestors_deleted_at and not is_owner
is_owner_or_admin = (is_owner or RoleChoices.ADMIN in roles) and not is_deleted
# Compute access roles before adding link roles because we don't
# want anonymous users to access versions (we wouldn't know from
# which date to allow them anyway)
# Anonymous users should also not see document accesses
has_role = bool(roles)
has_access_role = bool(roles) and not is_deleted
can_update_from_access = (
is_owner_or_admin or RoleChoices.EDITOR in roles
) and not is_deleted
# Add role provided by the document link
if self.link_reach == LinkReachChoices.PUBLIC or (
self.link_reach == LinkReachChoices.AUTHENTICATED and user.is_authenticated
):
roles.add(self.link_role)
is_owner_or_admin = bool(
roles.intersection({RoleChoices.OWNER, RoleChoices.ADMIN})
# Add roles provided by the document link, taking into account its ancestors
links_definitions = self.get_links_definitions(ancestors_links)
public_roles = links_definitions.get(LinkReachChoices.PUBLIC, set())
authenticated_roles = (
links_definitions.get(LinkReachChoices.AUTHENTICATED, set())
if user.is_authenticated
else set()
)
roles = roles | public_roles | authenticated_roles
can_get = bool(roles) and not is_deleted
can_update = (
is_owner_or_admin or RoleChoices.EDITOR in roles
) and not is_deleted
ai_allow_reach_from = settings.AI_ALLOW_REACH_FROM
ai_access = any(
[
ai_allow_reach_from == LinkReachChoices.PUBLIC and can_update,
ai_allow_reach_from == LinkReachChoices.AUTHENTICATED
and user.is_authenticated
and can_update,
ai_allow_reach_from == LinkReachChoices.RESTRICTED
and can_update_from_access,
]
)
can_get = bool(roles)
can_update = is_owner_or_admin or RoleChoices.EDITOR in roles
return {
"accesses_manage": is_owner_or_admin,
"accesses_view": has_role,
"ai_transform": can_update,
"ai_translate": can_update,
"accesses_view": has_access_role,
"ai_transform": ai_access,
"ai_translate": ai_access,
"attachment_upload": can_update,
"children_list": can_get,
"children_create": can_update and user.is_authenticated,
"collaboration_auth": can_get,
"destroy": RoleChoices.OWNER in roles,
"cors_proxy": can_get,
"descendants": can_get,
"destroy": is_owner,
"favorite": can_get and user.is_authenticated,
"link_configuration": is_owner_or_admin,
"invite_owner": RoleChoices.OWNER in roles,
"invite_owner": is_owner,
"move": is_owner_or_admin and not self.ancestors_deleted_at,
"partial_update": can_update,
"restore": is_owner,
"retrieve": can_get,
"media_auth": can_get,
"link_select_options": LinkReachChoices.get_select_options(ancestors_links),
"tree": can_get,
"update": can_update,
"versions_destroy": is_owner_or_admin,
"versions_list": has_role,
"versions_retrieve": has_role,
"versions_list": has_access_role,
"versions_retrieve": has_access_role,
}
def send_email(self, subject, emails, context=None, language=None):
@@ -596,6 +834,7 @@ class Document(BaseModel):
"document": self,
"domain": domain,
"link": f"{domain}/docs/{self.id}/",
"document_title": self.title or str(_("Untitled Document")),
"logo_img": settings.EMAIL_LOGO_IMG,
}
)
@@ -637,12 +876,91 @@ class Document(BaseModel):
'{name} invited you with the role "{role}" on the following document:'
).format(name=sender_name_email, role=role.lower()),
}
subject = _("{name} shared a document with you: {title}").format(
name=sender_name, title=self.title
subject = (
context["title"]
if not self.title
else _("{name} shared a document with you: {title}").format(
name=sender_name, title=self.title
)
)
self.send_email(subject, [email], context, language)
@transaction.atomic
def soft_delete(self):
"""
Soft delete the document, marking the deletion on descendants.
We still keep the .delete() method untouched for programmatic purposes.
"""
if (
self._meta.model.objects.filter(
models.Q(deleted_at__isnull=False)
| models.Q(ancestors_deleted_at__isnull=False),
pk=self.pk,
).exists()
or self.get_ancestors().filter(deleted_at__isnull=False).exists()
):
raise RuntimeError(
"This document is already deleted or has deleted ancestors."
)
self.ancestors_deleted_at = self.deleted_at = timezone.now()
self.save()
self.invalidate_nb_accesses_cache()
if self.depth > 1:
self._meta.model.objects.filter(pk=self.get_parent().pk).update(
numchild=models.F("numchild") - 1,
has_deleted_children=True,
)
# Mark all descendants as soft deleted
self.get_descendants().filter(ancestors_deleted_at__isnull=True).update(
ancestors_deleted_at=self.ancestors_deleted_at
)
@transaction.atomic
def restore(self):
"""Cancelling a soft delete with checks."""
# This should not happen
if self._meta.model.objects.filter(
pk=self.pk, deleted_at__isnull=True
).exists():
raise RuntimeError("This document is not deleted.")
if self.deleted_at < get_trashbin_cutoff():
raise RuntimeError(
"This document was permanently deleted and cannot be restored."
)
# save the current deleted_at value to exclude it from the descendants update
current_deleted_at = self.deleted_at
# Restore the current document
self.deleted_at = None
# Calculate the minimum `deleted_at` among all ancestors
ancestors_deleted_at = (
self.get_ancestors()
.filter(deleted_at__isnull=False)
.order_by("deleted_at")
.values_list("deleted_at", flat=True)
.first()
)
self.ancestors_deleted_at = ancestors_deleted_at
self.save(update_fields=["deleted_at", "ancestors_deleted_at"])
self.invalidate_nb_accesses_cache()
self.get_descendants().exclude(
models.Q(deleted_at__isnull=False)
| models.Q(ancestors_deleted_at__lt=current_deleted_at)
).update(ancestors_deleted_at=self.ancestors_deleted_at)
if self.depth > 1:
self._meta.model.objects.filter(pk=self.get_parent().pk).update(
numchild=models.F("numchild") + 1
)
class LinkTrace(BaseModel):
"""
@@ -745,6 +1063,16 @@ class DocumentAccess(BaseAccess):
def __str__(self):
return f"{self.user!s} is {self.role:s} in document {self.document!s}"
def save(self, *args, **kwargs):
"""Override save to clear the document's cache for number of accesses."""
super().save(*args, **kwargs)
self.document.invalidate_nb_accesses_cache()
def delete(self, *args, **kwargs):
"""Override delete to clear the document's cache for number of accesses."""
super().delete(*args, **kwargs)
self.document.invalidate_nb_accesses_cache()
def get_abilities(self, user):
"""
Compute and return abilities for a given user on the document access.
@@ -774,11 +1102,27 @@ class Template(BaseModel):
def __str__(self):
return self.title
def get_roles(self, user):
"""Return the roles a user has on a resource as an iterable."""
if not user.is_authenticated:
return []
try:
roles = self.user_roles or []
except AttributeError:
try:
roles = self.accesses.filter(
models.Q(user=user) | models.Q(team__in=user.teams),
).values_list("role", flat=True)
except (models.ObjectDoesNotExist, IndexError):
roles = []
return roles
def get_abilities(self, user):
"""
Compute and return abilities for a given user on the template.
"""
roles = get_resource_roles(self, user)
roles = self.get_roles(user)
is_owner_or_admin = bool(
set(roles).intersection({RoleChoices.OWNER, RoleChoices.ADMIN})
)
@@ -794,107 +1138,6 @@ class Template(BaseModel):
"retrieve": can_get,
}
def generate_pdf(self, body_html, metadata):
"""
Generate and return a pdf document wrapped around the current template
"""
document_html = weasyprint.HTML(
string=DjangoTemplate(self.code).render(
Context({"body": html.format_html(body_html), **metadata})
)
)
css = weasyprint.CSS(
string=self.css,
font_config=weasyprint.text.fonts.FontConfiguration(),
)
pdf_content = document_html.write_pdf(stylesheets=[css], zoom=1)
response = FileResponse(BytesIO(pdf_content), content_type="application/pdf")
response["Content-Disposition"] = f"attachment; filename={self.title}.pdf"
return response
def generate_word(self, body_html, metadata):
"""
Generate and return a docx document wrapped around the current template
"""
template_string = DjangoTemplate(self.code).render(
Context({"body": html.format_html(body_html), **metadata})
)
html_string = f"""
<!DOCTYPE html>
<html>
<head>
<style>
{self.css}
</style>
</head>
<body>
{template_string}
</body>
</html>
"""
reference_docx = "core/static/reference.docx"
output = BytesIO()
# Convert the HTML to a temporary docx file
with tempfile.NamedTemporaryFile(suffix=".docx", prefix="docx_") as tmp_file:
output_path = tmp_file.name
pypandoc.convert_text(
html_string,
"docx",
format="html",
outputfile=output_path,
extra_args=["--reference-doc", reference_docx],
)
# Create a BytesIO object to store the output of the temporary docx file
with open(output_path, "rb") as f:
output = BytesIO(f.read())
# Ensure the pointer is at the beginning
output.seek(0)
response = FileResponse(
output,
content_type="application/vnd.openxmlformats-officedocument.wordprocessingml.document",
)
response["Content-Disposition"] = f"attachment; filename={self.title}.docx"
return response
def generate_document(self, body, body_type, export_format):
"""
Generate and return a document for this template around the
body passed as argument.
2 types of body are accepted:
- HTML: body_type = "html"
- Markdown: body_type = "markdown"
2 types of documents can be generated:
- PDF: export_format = "pdf"
- Docx: export_format = "docx"
"""
document = frontmatter.loads(body)
metadata = document.metadata
strip_body = document.content.strip()
if body_type == "html":
body_html = strip_body
else:
body_html = (
markdown.markdown(textwrap.dedent(strip_body)) if strip_body else ""
)
if export_format == "pdf":
return self.generate_pdf(body_html, metadata)
return self.generate_word(body_html, metadata)
class TemplateAccess(BaseAccess):
"""Relation model to give access to a template for a user or a team with a role."""
@@ -983,8 +1226,8 @@ class Invitation(BaseModel):
User.objects.filter(email=self.email).exists()
and not settings.OIDC_ALLOW_DUPLICATE_EMAILS
):
raise exceptions.ValidationError(
{"email": _("This email is already associated to a registered user.")}
raise ValidationError(
{"email": [_("This email is already associated to a registered user.")]}
)
@property

View File

@@ -1,8 +1,5 @@
"""AI services."""
import json
import re
from django.conf import settings
from django.core.exceptions import ImproperlyConfigured
@@ -12,32 +9,44 @@ from core import enums
AI_ACTIONS = {
"prompt": (
"Answer the prompt in markdown format. Return JSON: "
'{"answer": "Your markdown answer"}. '
"Do not provide any other information."
"Answer the prompt in markdown format. "
"Preserve the language and markdown formatting. "
"Do not provide any other information. "
"Preserve the language."
),
"correct": (
"Correct grammar and spelling of the markdown text, "
"preserving language and markdown formatting. "
'Return JSON: {"answer": "your corrected markdown text"}. '
"Do not provide any other information."
"Do not provide any other information. "
"Preserve the language."
),
"rephrase": (
"Rephrase the given markdown text, "
"preserving language and markdown formatting. "
'Return JSON: {"answer": "your rephrased markdown text"}. '
"Do not provide any other information."
"Do not provide any other information. "
"Preserve the language."
),
"summarize": (
"Summarize the markdown text, preserving language and markdown formatting. "
'Return JSON: {"answer": "your markdown summary"}. '
"Do not provide any other information."
"Do not provide any other information. "
"Preserve the language."
),
"beautify": (
"Add formatting to the text to make it more readable. "
"Do not provide any other information. "
"Preserve the language."
),
"emojify": (
"Add emojis to the important parts of the text. "
"Do not provide any other information. "
"Preserve the language."
),
}
AI_TRANSLATE = (
"Translate the markdown text to {language:s}, preserving markdown formatting. "
'Return JSON: {{"answer": "your translated markdown text in {language:s}"}}. '
"Keep the same html stucture and formatting. "
"Translate the content in the html to the specified language {language:s}. "
"Check the translation for accuracy and make any necessary corrections. "
"Do not provide any other information."
)
@@ -59,32 +68,18 @@ class AIService:
"""Helper method to call the OpenAI API and process the response."""
response = self.client.chat.completions.create(
model=settings.AI_MODEL,
response_format={"type": "json_object"},
messages=[
{"role": "system", "content": system_content},
{"role": "user", "content": json.dumps({"markdown_input": text})},
{"role": "user", "content": text},
],
)
content = response.choices[0].message.content
try:
sanitized_content = re.sub(r'\s*"answer"\s*:\s*', '"answer": ', content)
sanitized_content = re.sub(r"\s*\}", "}", sanitized_content)
sanitized_content = re.sub(r"(?<!\\)\n", "\\\\n", sanitized_content)
sanitized_content = re.sub(r"(?<!\\)\t", "\\\\t", sanitized_content)
json_response = json.loads(sanitized_content)
except (json.JSONDecodeError, IndexError):
try:
json_response = json.loads(content)
except json.JSONDecodeError as err:
raise RuntimeError("AI response is not valid JSON", content) from err
if "answer" not in json_response:
if not content:
raise RuntimeError("AI response does not contain an answer")
return json_response
return {"answer": content}
def transform(self, text, action):
"""Transform text based on specified action."""

View File

@@ -0,0 +1,50 @@
"""
Unit test for `update_files_content_type_metadata` command.
"""
import uuid
from django.core.files.storage import default_storage
from django.core.management import call_command
import pytest
from core import factories
@pytest.mark.django_db
def test_update_files_content_type_metadata():
"""
Test that the command `update_files_content_type_metadata`
fixes the ContentType of attachment in the storage.
"""
s3_client = default_storage.connection.meta.client
bucket_name = default_storage.bucket_name
# Create files with a wrong ContentType
keys = []
for _ in range(10):
doc_id = uuid.uuid4()
factories.DocumentFactory(id=doc_id)
key = f"{doc_id}/attachments/testfile.png"
keys.append(key)
fake_png = b"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR..."
s3_client.put_object(
Bucket=bucket_name,
Key=key,
Body=fake_png,
ContentType="text/plain",
Metadata={"owner": "None"},
)
# Call the command that fixes the ContentType
call_command("update_files_content_type_metadata")
for key in keys:
head_resp = s3_client.head_object(Bucket=bucket_name, Key=key)
assert head_resp["ContentType"] == "image/png", (
f"ContentType not fixed, got {head_resp['ContentType']!r}"
)
# Check that original metadata was preserved
assert head_resp["Metadata"].get("owner") == "None"

View File

@@ -76,14 +76,14 @@ def test_api_document_accesses_list_authenticated_related(via, mock_user_teams):
user_access = models.DocumentAccess.objects.create(
document=document,
user=user,
role=random.choice(models.RoleChoices.choices)[0],
role=random.choice(models.RoleChoices.values),
)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
user_access = models.DocumentAccess.objects.create(
document=document,
team="lasuite",
role=random.choice(models.RoleChoices.choices)[0],
role=random.choice(models.RoleChoices.values),
)
access1 = factories.TeamDocumentAccessFactory(document=document)
@@ -227,7 +227,7 @@ def test_api_document_accesses_update_anonymous():
new_values = {
"id": uuid4(),
"user": factories.UserFactory().id,
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
api_client = APIClient()
@@ -260,7 +260,7 @@ def test_api_document_accesses_update_authenticated_unrelated():
new_values = {
"id": uuid4(),
"user": factories.UserFactory().id,
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
for field, value in new_values.items():
@@ -302,7 +302,7 @@ def test_api_document_accesses_update_authenticated_reader_or_editor(
new_values = {
"id": uuid4(),
"user": factories.UserFactory().id,
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
for field, value in new_values.items():
@@ -413,7 +413,7 @@ def test_api_document_accesses_update_administrator_from_owner(via, mock_user_te
new_values = {
"id": uuid4(),
"user_id": factories.UserFactory().id,
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
for field, value in new_values.items():
@@ -527,7 +527,7 @@ def test_api_document_accesses_update_owner(
new_values = {
"id": uuid4(),
"user_id": factories.UserFactory().id,
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
for field, value in new_values.items():

View File

@@ -16,6 +16,9 @@ from core.tests.conftest import TEAM, USER, VIA
pytestmark = pytest.mark.django_db
# Create
def test_api_document_accesses_create_anonymous():
"""Anonymous users should not be allowed to create document accesses."""
document = factories.DocumentFactory()
@@ -26,7 +29,7 @@ def test_api_document_accesses_create_anonymous():
{
"user_id": str(other_user.id),
"document": str(document.id),
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
},
format="json",
)
@@ -123,7 +126,7 @@ def test_api_document_accesses_create_authenticated_administrator(via, mock_user
document=document, team="lasuite", role="administrator"
)
other_user = factories.UserFactory()
other_user = factories.UserFactory(language="en-us")
# It should not be allowed to create an owner access
response = client.post(
@@ -199,7 +202,7 @@ def test_api_document_accesses_create_authenticated_owner(via, mock_user_teams):
document=document, team="lasuite", role="owner"
)
other_user = factories.UserFactory()
other_user = factories.UserFactory(language="en-us")
role = random.choice([role[0] for role in models.RoleChoices.choices])
@@ -235,3 +238,73 @@ def test_api_document_accesses_create_authenticated_owner(via, mock_user_teams):
f"on the following document: {document.title}"
) in email_content
assert "docs/" + str(document.id) + "/" in email_content
@pytest.mark.parametrize("via", VIA)
def test_api_document_accesses_create_email_in_receivers_language(via, mock_user_teams):
"""
The email sent to the accesses to notify them of the adding, should be in their language.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory()
if via == USER:
factories.UserDocumentAccessFactory(document=document, user=user, role="owner")
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamDocumentAccessFactory(
document=document, team="lasuite", role="owner"
)
role = random.choice([role[0] for role in models.RoleChoices.choices])
assert len(mail.outbox) == 0
other_users = (
factories.UserFactory(language="en-us"),
factories.UserFactory(language="fr-fr"),
)
for index, other_user in enumerate(other_users):
expected_language = other_user.language
response = client.post(
f"/api/v1.0/documents/{document.id!s}/accesses/",
{
"user_id": str(other_user.id),
"role": role,
},
format="json",
)
assert response.status_code == 201
assert models.DocumentAccess.objects.filter(user=other_user).count() == 1
new_document_access = models.DocumentAccess.objects.filter(
user=other_user
).get()
other_user_data = serializers.UserSerializer(instance=other_user).data
assert response.json() == {
"id": str(new_document_access.id),
"user": other_user_data,
"team": "",
"role": role,
"abilities": new_document_access.get_abilities(user),
}
assert len(mail.outbox) == index + 1
email = mail.outbox[index]
assert email.to == [other_user_data["email"]]
email_content = " ".join(email.body.split())
email_subject = " ".join(email.subject.split())
if expected_language == "en-us":
assert (
f"{user.full_name} shared a document with you: {document.title}".lower()
in email_subject.lower()
)
elif expected_language == "fr-fr":
assert (
f"{user.full_name} a partagé un document avec vous: {document.title}".lower()
in email_subject.lower()
)
assert "docs/" + str(document.id) + "/" in email_content.lower()

View File

@@ -304,7 +304,7 @@ def test_api_document_invitations_create_anonymous():
document = factories.DocumentFactory()
invitation_values = {
"email": "guest@example.com",
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
response = APIClient().post(
@@ -325,7 +325,7 @@ def test_api_document_invitations_create_authenticated_outsider():
document = factories.DocumentFactory()
invitation_values = {
"email": "guest@example.com",
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
client = APIClient()
@@ -370,7 +370,7 @@ def test_api_document_invitations_create_privileged_members(
Only owners and administrators should be able to invite new users.
Only owners can invite owners.
"""
user = factories.UserFactory()
user = factories.UserFactory(language="en-us")
document = factories.DocumentFactory()
if via == USER:
factories.UserDocumentAccessFactory(document=document, user=user, role=inviting)
@@ -422,11 +422,12 @@ def test_api_document_invitations_create_privileged_members(
}
def test_api_document_invitations_create_email_from_content_language():
def test_api_document_invitations_create_email_from_senders_language():
"""
The email generated is from the language set in the Content-Language header
When inviting on a document a user who does not exist yet in our database,
the invitation email should be sent in the language of the sending user.
"""
user = factories.UserFactory()
user = factories.UserFactory(language="fr-fr")
document = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=document, user=user, role="owner")
@@ -444,7 +445,6 @@ def test_api_document_invitations_create_email_from_content_language():
f"/api/v1.0/documents/{document.id!s}/invitations/",
invitation_values,
format="json",
headers={"Content-Language": "fr-fr"},
)
assert response.status_code == 201
@@ -458,52 +458,17 @@ def test_api_document_invitations_create_email_from_content_language():
email_content = " ".join(email.body.split())
assert f"{user.full_name} a partagé un document avec vous!" in email_content
def test_api_document_invitations_create_email_from_content_language_not_supported():
"""
If the language from the Content-Language is not supported
it will display the default language, English.
"""
user = factories.UserFactory()
document = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=document, user=user, role="owner")
invitation_values = {
"email": "guest@example.com",
"role": "reader",
}
assert len(mail.outbox) == 0
client = APIClient()
client.force_login(user)
response = client.post(
f"/api/v1.0/documents/{document.id!s}/invitations/",
invitation_values,
format="json",
headers={"Content-Language": "not-supported"},
assert (
"Docs, votre nouvel outil incontournable pour organiser, partager et collaborer "
"sur vos documents en équipe." in email_content
)
assert response.status_code == 201
assert response.json()["email"] == "guest@example.com"
assert models.Invitation.objects.count() == 1
assert len(mail.outbox) == 1
email = mail.outbox[0]
assert email.to == ["guest@example.com"]
email_content = " ".join(email.body.split())
assert f"{user.full_name} shared a document with you!" in email_content
def test_api_document_invitations_create_email_full_name_empty():
"""
If the full name of the user is empty, it will display the email address.
"""
user = factories.UserFactory(full_name="")
user = factories.UserFactory(full_name="", language="en-us")
document = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=document, user=user, role="owner")
@@ -550,7 +515,7 @@ def test_api_document_invitations_create_issuer_and_document_override():
"document": str(other_document.id),
"issuer": str(factories.UserFactory().id),
"email": "guest@example.com",
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
client = APIClient()
@@ -595,9 +560,11 @@ def test_api_document_invitations_create_cannot_duplicate_invitation():
)
assert response.status_code == 400
assert response.json() == [
"Document invitation with this Email address and Document already exists."
]
assert response.json() == {
"__all__": [
"Document invitation with this Email address and Document already exists."
],
}
def test_api_document_invitations_create_cannot_invite_existing_users():
@@ -611,7 +578,7 @@ def test_api_document_invitations_create_cannot_invite_existing_users():
# Build an invitation to the email of an exising identity in the db
invitation_values = {
"email": existing_user.email,
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
client = APIClient()
@@ -624,7 +591,9 @@ def test_api_document_invitations_create_cannot_invite_existing_users():
)
assert response.status_code == 400
assert response.json() == ["This email is already associated to a registered user."]
assert response.json() == {
"email": ["This email is already associated to a registered user."]
}
# Update

View File

@@ -75,14 +75,14 @@ def test_api_document_versions_list_authenticated_related_success(via, mock_user
models.DocumentAccess.objects.create(
document=document,
user=user,
role=random.choice(models.RoleChoices.choices)[0],
role=random.choice(models.RoleChoices.values),
)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
models.DocumentAccess.objects.create(
document=document,
team="lasuite",
role=random.choice(models.RoleChoices.choices)[0],
role=random.choice(models.RoleChoices.values),
)
# Other versions of documents to which the user has access should not be listed
@@ -134,14 +134,14 @@ def test_api_document_versions_list_authenticated_related_pagination(
models.DocumentAccess.objects.create(
document=document,
user=user,
role=random.choice(models.RoleChoices.choices)[0],
role=random.choice(models.RoleChoices.values),
)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
models.DocumentAccess.objects.create(
document=document,
team="lasuite",
role=random.choice(models.RoleChoices.choices)[0],
role=random.choice(models.RoleChoices.values),
)
for i in range(4):
@@ -185,6 +185,84 @@ def test_api_document_versions_list_authenticated_related_pagination(
assert content["versions"][0]["version_id"] == all_version_ids[2]
@pytest.mark.parametrize("via", VIA)
def test_api_document_versions_list_authenticated_related_pagination_parent(
via, mock_user_teams
):
"""
When a user gains access to a document's versions via an ancestor, the date of access
to the parent should be used to filter versions that were created prior to the
user gaining access to the document.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
grand_parent = factories.DocumentFactory()
parent = factories.DocumentFactory(parent=grand_parent)
document = factories.DocumentFactory(parent=parent)
for i in range(3):
document.content = f"before {i:d}"
document.save()
if via == USER:
models.DocumentAccess.objects.create(
document=grand_parent,
user=user,
role=random.choice(models.RoleChoices.values),
)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
models.DocumentAccess.objects.create(
document=grand_parent,
team="lasuite",
role=random.choice(models.RoleChoices.values),
)
for i in range(4):
document.content = f"after {i:d}"
document.save()
response = client.get(
f"/api/v1.0/documents/{document.id!s}/versions/",
)
content = response.json()
assert response.status_code == 200
assert content["is_truncated"] is False
# The current version is not listed
assert content["count"] == 3
assert content["next_version_id_marker"] == ""
all_version_ids = [version["version_id"] for version in content["versions"]]
# - set page size
response = client.get(
f"/api/v1.0/documents/{document.id!s}/versions/?page_size=2",
)
content = response.json()
assert content["count"] == 2
assert content["is_truncated"] is True
marker = content["next_version_id_marker"]
assert marker == all_version_ids[1]
assert [
version["version_id"] for version in content["versions"]
] == all_version_ids[:2]
# - get page 2
response = client.get(
f"/api/v1.0/documents/{document.id!s}/versions/?page_size=2&version_id={marker:s}",
)
content = response.json()
assert content["count"] == 1
assert content["is_truncated"] is False
assert content["next_version_id_marker"] == ""
assert content["versions"][0]["version_id"] == all_version_ids[2]
def test_api_document_versions_list_exceeds_max_page_size():
"""Page size should not exceed the limit set on the serializer"""
user = factories.UserFactory()
@@ -314,6 +392,74 @@ def test_api_document_versions_retrieve_authenticated_related(via, mock_user_tea
assert response.json()["content"] == "new content 1"
@pytest.mark.parametrize("via", VIA)
def test_api_document_versions_retrieve_authenticated_related_parent(
via, mock_user_teams
):
"""
A user who gains access to a document's versions via one of its ancestors, should be able to
retrieve the document versions. The date of access to the parent should be used to filter
versions that were created prior to the user gaining access to the document.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
grand_parent = factories.DocumentFactory()
parent = factories.DocumentFactory(parent=grand_parent)
document = factories.DocumentFactory(parent=parent)
document.content = "new content"
document.save()
assert len(document.get_versions_slice()["versions"]) == 1
version_id = document.get_versions_slice()["versions"][0]["version_id"]
if via == USER:
factories.UserDocumentAccessFactory(document=grand_parent, user=user)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamDocumentAccessFactory(document=grand_parent, team="lasuite")
time.sleep(1) # minio stores datetimes with the precision of a second
# Versions created before the document was shared should not be seen by the user
response = client.get(
f"/api/v1.0/documents/{document.id!s}/versions/{version_id:s}/",
)
assert response.status_code == 404
# Create a new version should not make it available to the user because
# only the current version is available to the user but it is excluded
# from the list
document.content = "new content 1"
document.save()
assert len(document.get_versions_slice()["versions"]) == 2
version_id = document.get_versions_slice()["versions"][0]["version_id"]
response = client.get(
f"/api/v1.0/documents/{document.id!s}/versions/{version_id:s}/",
)
assert response.status_code == 404
# Adding one more version should make the previous version available to the user
document.content = "new content 2"
document.save()
assert len(document.get_versions_slice()["versions"]) == 3
version_id = document.get_versions_slice()["versions"][0]["version_id"]
response = client.get(
f"/api/v1.0/documents/{document.id!s}/versions/{version_id:s}/",
)
assert response.status_code == 200
assert response.json()["content"] == "new content 1"
def test_api_document_versions_create_anonymous():
"""Anonymous users should not be allowed to create document versions."""
document = factories.DocumentFactory()
@@ -458,15 +604,19 @@ def test_api_document_versions_update_authenticated_related(via, mock_user_teams
# Delete
def test_api_document_versions_delete_anonymous():
@pytest.mark.parametrize("reach", models.LinkReachChoices.values)
def test_api_document_versions_delete_anonymous(reach):
"""Anonymous users should not be allowed to destroy a document version."""
access = factories.UserDocumentAccessFactory()
access = factories.UserDocumentAccessFactory(document__link_reach=reach)
response = APIClient().delete(
f"/api/v1.0/documents/{access.document_id!s}/versions/{access.id!s}/",
)
assert response.status_code == 401
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
@pytest.mark.parametrize("reach", models.LinkReachChoices.values)

View File

@@ -2,6 +2,7 @@
Test AI transform API endpoint for users in impress's core app.
"""
import random
from unittest.mock import MagicMock, patch
from django.core.cache import cache
@@ -31,6 +32,9 @@ def ai_settings():
yield
@override_settings(
AI_ALLOW_REACH_FROM=random.choice(["public", "authenticated", "restricted"])
)
@pytest.mark.parametrize(
"reach, role",
[
@@ -57,6 +61,7 @@ def test_api_documents_ai_transform_anonymous_forbidden(reach, role):
}
@override_settings(AI_ALLOW_REACH_FROM="public")
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_transform_anonymous_success(mock_create):
@@ -66,6 +71,40 @@ def test_api_documents_ai_transform_anonymous_success(mock_create):
"""
document = factories.DocumentFactory(link_reach="public", link_role="editor")
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content="Salut"))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
response = APIClient().post(url, {"text": "Hello", "action": "summarize"})
assert response.status_code == 200
assert response.json() == {"answer": "Salut"}
mock_create.assert_called_once_with(
model="llama",
messages=[
{
"role": "system",
"content": (
"Summarize the markdown text, preserving language and markdown formatting. "
"Do not provide any other information. Preserve the language."
),
},
{"role": "user", "content": "Hello"},
],
)
@override_settings(AI_ALLOW_REACH_FROM=random.choice(["authenticated", "restricted"]))
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_transform_anonymous_limited_by_setting(mock_create):
"""
Anonymous users should be able to request AI transform to a document
if the link reach and role permit it.
"""
document = factories.DocumentFactory(link_reach="public", link_role="editor")
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
@@ -74,23 +113,7 @@ def test_api_documents_ai_transform_anonymous_success(mock_create):
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
response = APIClient().post(url, {"text": "Hello", "action": "summarize"})
assert response.status_code == 200
assert response.json() == {"answer": "Salut"}
mock_create.assert_called_once_with(
model="llama",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": (
"Summarize the markdown text, preserving language and markdown formatting. "
'Return JSON: {"answer": "your markdown summary"}. Do not provide any other '
"information."
),
},
{"role": "user", "content": '{"markdown_input": "Hello"}'},
],
)
assert response.status_code == 401
@pytest.mark.parametrize(
@@ -144,9 +167,8 @@ def test_api_documents_ai_transform_authenticated_success(mock_create, reach, ro
document = factories.DocumentFactory(link_reach=reach, link_role=role)
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
choices=[MagicMock(message=MagicMock(content="Salut"))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
@@ -156,16 +178,15 @@ def test_api_documents_ai_transform_authenticated_success(mock_create, reach, ro
assert response.json() == {"answer": "Salut"}
mock_create.assert_called_once_with(
model="llama",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": (
'Answer the prompt in markdown format. Return JSON: {"answer": '
'"Your markdown answer"}. Do not provide any other information.'
"Answer the prompt in markdown format. Preserve the language and markdown "
"formatting. Do not provide any other information. Preserve the language."
),
},
{"role": "user", "content": '{"markdown_input": "Hello"}'},
{"role": "user", "content": "Hello"},
],
)
@@ -220,9 +241,8 @@ def test_api_documents_ai_transform_success(mock_create, via, role, mock_user_te
document=document, team="lasuite", role=role
)
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
choices=[MagicMock(message=MagicMock(content="Salut"))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
@@ -232,16 +252,15 @@ def test_api_documents_ai_transform_success(mock_create, via, role, mock_user_te
assert response.json() == {"answer": "Salut"}
mock_create.assert_called_once_with(
model="llama",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": (
'Answer the prompt in markdown format. Return JSON: {"answer": '
'"Your markdown answer"}. Do not provide any other information.'
"Answer the prompt in markdown format. Preserve the language and markdown "
"formatting. Do not provide any other information. Preserve the language."
),
},
{"role": "user", "content": '{"markdown_input": "Hello"}'},
{"role": "user", "content": "Hello"},
],
)
@@ -289,9 +308,8 @@ def test_api_documents_ai_transform_throttling_document(mock_create):
client = APIClient()
document = factories.DocumentFactory(link_reach="public", link_role="editor")
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
choices=[MagicMock(message=MagicMock(content="Salut"))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-transform/"
@@ -324,9 +342,8 @@ def test_api_documents_ai_transform_throttling_user(mock_create):
client = APIClient()
client.force_login(user)
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
choices=[MagicMock(message=MagicMock(content="Salut"))]
)
for _ in range(3):

View File

@@ -2,6 +2,7 @@
Test AI translate API endpoint for users in impress's core app.
"""
import random
from unittest.mock import MagicMock, patch
from django.core.cache import cache
@@ -51,6 +52,9 @@ def test_api_documents_ai_translate_viewset_options_metadata():
}
@override_settings(
AI_ALLOW_REACH_FROM=random.choice(["public", "authenticated", "restricted"])
)
@pytest.mark.parametrize(
"reach, role",
[
@@ -77,6 +81,7 @@ def test_api_documents_ai_translate_anonymous_forbidden(reach, role):
}
@override_settings(AI_ALLOW_REACH_FROM="public")
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_translate_anonymous_success(mock_create):
@@ -86,6 +91,42 @@ def test_api_documents_ai_translate_anonymous_success(mock_create):
"""
document = factories.DocumentFactory(link_reach="public", link_role="editor")
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content="Ola"))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
response = APIClient().post(url, {"text": "Hello", "language": "es"})
assert response.status_code == 200
assert response.json() == {"answer": "Ola"}
mock_create.assert_called_once_with(
model="llama",
messages=[
{
"role": "system",
"content": (
"Keep the same html stucture and formatting. "
"Translate the content in the html to the specified language Spanish. "
"Check the translation for accuracy and make any necessary corrections. "
"Do not provide any other information."
),
},
{"role": "user", "content": "Hello"},
],
)
@override_settings(AI_ALLOW_REACH_FROM=random.choice(["authenticated", "restricted"]))
@pytest.mark.usefixtures("ai_settings")
@patch("openai.resources.chat.completions.Completions.create")
def test_api_documents_ai_translate_anonymous_limited_by_setting(mock_create):
"""
Anonymous users should be able to request AI translate to a document
if the link reach and role permit it.
"""
document = factories.DocumentFactory(link_reach="public", link_role="editor")
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
@@ -94,23 +135,7 @@ def test_api_documents_ai_translate_anonymous_success(mock_create):
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
response = APIClient().post(url, {"text": "Hello", "language": "es"})
assert response.status_code == 200
assert response.json() == {"answer": "Salut"}
mock_create.assert_called_once_with(
model="llama",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": (
"Translate the markdown text to Spanish, preserving markdown formatting. "
'Return JSON: {"answer": "your translated markdown text in Spanish"}. '
"Do not provide any other information."
),
},
{"role": "user", "content": '{"markdown_input": "Hello"}'},
],
)
assert response.status_code == 401
@pytest.mark.parametrize(
@@ -164,9 +189,8 @@ def test_api_documents_ai_translate_authenticated_success(mock_create, reach, ro
document = factories.DocumentFactory(link_reach=reach, link_role=role)
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
choices=[MagicMock(message=MagicMock(content="Salut"))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
@@ -176,18 +200,18 @@ def test_api_documents_ai_translate_authenticated_success(mock_create, reach, ro
assert response.json() == {"answer": "Salut"}
mock_create.assert_called_once_with(
model="llama",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": (
"Translate the markdown text to Colombian Spanish, "
"preserving markdown formatting. Return JSON: "
'{"answer": "your translated markdown text in Colombian Spanish"}. '
"Keep the same html stucture and formatting. "
"Translate the content in the html to the "
"specified language Colombian Spanish. "
"Check the translation for accuracy and make any necessary corrections. "
"Do not provide any other information."
),
},
{"role": "user", "content": '{"markdown_input": "Hello"}'},
{"role": "user", "content": "Hello"},
],
)
@@ -242,9 +266,8 @@ def test_api_documents_ai_translate_success(mock_create, via, role, mock_user_te
document=document, team="lasuite", role=role
)
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
choices=[MagicMock(message=MagicMock(content="Salut"))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
@@ -254,18 +277,18 @@ def test_api_documents_ai_translate_success(mock_create, via, role, mock_user_te
assert response.json() == {"answer": "Salut"}
mock_create.assert_called_once_with(
model="llama",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": (
"Translate the markdown text to Colombian Spanish, "
"preserving markdown formatting. Return JSON: "
'{"answer": "your translated markdown text in Colombian Spanish"}. '
"Keep the same html stucture and formatting. "
"Translate the content in the html to the "
"specified language Colombian Spanish. "
"Check the translation for accuracy and make any necessary corrections. "
"Do not provide any other information."
),
},
{"role": "user", "content": '{"markdown_input": "Hello"}'},
{"role": "user", "content": "Hello"},
],
)
@@ -313,9 +336,8 @@ def test_api_documents_ai_translate_throttling_document(mock_create):
client = APIClient()
document = factories.DocumentFactory(link_reach="public", link_role="editor")
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
choices=[MagicMock(message=MagicMock(content="Salut"))]
)
url = f"/api/v1.0/documents/{document.id!s}/ai-translate/"
@@ -348,9 +370,8 @@ def test_api_documents_ai_translate_throttling_user(mock_create):
client = APIClient()
client.force_login(user)
answer = '{"answer": "Salut"}'
mock_create.return_value = MagicMock(
choices=[MagicMock(message=MagicMock(content=answer))]
choices=[MagicMock(message=MagicMock(content="Salut"))]
)
for _ in range(3):

View File

@@ -64,12 +64,23 @@ def test_api_documents_attachment_upload_anonymous_success():
assert response.status_code == 201
pattern = re.compile(rf"^/media/{document.id!s}/attachments/(.*)\.png")
match = pattern.search(response.json()["file"])
file_path = response.json()["file"]
match = pattern.search(file_path)
file_id = match.group(1)
# Validate that file_id is a valid UUID
uuid.UUID(file_id)
# Now, check the metadata of the uploaded file
key = file_path.replace("/media", "")
file_head = default_storage.connection.meta.client.head_object(
Bucket=default_storage.bucket_name, Key=key
)
assert file_head["Metadata"] == {"owner": "None"}
assert file_head["ContentType"] == "image/png"
assert file_head["ContentDisposition"] == 'inline; filename="test.png"'
@pytest.mark.parametrize(
"reach, role",
@@ -206,6 +217,8 @@ def test_api_documents_attachment_upload_success(via, role, mock_user_teams):
Bucket=default_storage.bucket_name, Key=key
)
assert file_head["Metadata"] == {"owner": str(user.id)}
assert file_head["ContentType"] == "image/png"
assert file_head["ContentDisposition"] == 'inline; filename="test.png"'
def test_api_documents_attachment_upload_invalid(client):
@@ -247,16 +260,18 @@ def test_api_documents_attachment_upload_size_limit_exceeded(settings):
@pytest.mark.parametrize(
"name,content,extension",
"name,content,extension,content_type",
[
("test.exe", b"text", "exe"),
("test", b"text", "txt"),
("test.aaaaaa", b"test", "txt"),
("test.txt", PIXEL, "txt"),
("test.py", b"#!/usr/bin/python", "py"),
("test.exe", b"text", "exe", "text/plain"),
("test", b"text", "txt", "text/plain"),
("test.aaaaaa", b"test", "txt", "text/plain"),
("test.txt", PIXEL, "txt", "image/png"),
("test.py", b"#!/usr/bin/python", "py", "text/plain"),
],
)
def test_api_documents_attachment_upload_fix_extension(name, content, extension):
def test_api_documents_attachment_upload_fix_extension(
name, content, extension, content_type
):
"""
A file with no extension or a wrong extension is accepted and the extension
is corrected in storage.
@@ -278,7 +293,9 @@ def test_api_documents_attachment_upload_fix_extension(name, content, extension)
match = pattern.search(file_path)
file_id = match.group(1)
assert "-unsafe" in file_id
# Validate that file_id is a valid UUID
file_id = file_id.replace("-unsafe", "")
uuid.UUID(file_id)
# Now, check the metadata of the uploaded file
@@ -287,6 +304,8 @@ def test_api_documents_attachment_upload_fix_extension(name, content, extension)
Bucket=default_storage.bucket_name, Key=key
)
assert file_head["Metadata"] == {"owner": str(user.id), "is_unsafe": "true"}
assert file_head["ContentType"] == content_type
assert file_head["ContentDisposition"] == f'attachment; filename="{name:s}"'
def test_api_documents_attachment_upload_empty_file():
@@ -326,7 +345,9 @@ def test_api_documents_attachment_upload_unsafe():
match = pattern.search(file_path)
file_id = match.group(1)
assert "-unsafe" in file_id
# Validate that file_id is a valid UUID
file_id = file_id.replace("-unsafe", "")
uuid.UUID(file_id)
# Now, check the metadata of the uploaded file
@@ -335,3 +356,5 @@ def test_api_documents_attachment_upload_unsafe():
Bucket=default_storage.bucket_name, Key=key
)
assert file_head["Metadata"] == {"owner": str(user.id), "is_unsafe": "true"}
assert file_head["ContentType"] == "application/octet-stream"
assert file_head["ContentDisposition"] == 'attachment; filename="script.exe"'

View File

@@ -0,0 +1,251 @@
"""
Tests for Documents API endpoint in impress's core app: children create
"""
from uuid import uuid4
import pytest
from rest_framework.test import APIClient
from core import factories
from core.models import Document, LinkReachChoices, LinkRoleChoices
pytestmark = pytest.mark.django_db
@pytest.mark.parametrize("depth", [1, 2, 3])
@pytest.mark.parametrize("role", LinkRoleChoices.values)
@pytest.mark.parametrize("reach", LinkReachChoices.values)
def test_api_documents_children_create_anonymous(reach, role, depth):
"""Anonymous users should not be allowed to create children documents."""
for i in range(depth):
if i == 0:
document = factories.DocumentFactory(link_reach=reach, link_role=role)
else:
document = factories.DocumentFactory(parent=document)
response = APIClient().post(
f"/api/v1.0/documents/{document.id!s}/children/",
{
"title": "my document",
},
)
assert Document.objects.count() == depth
assert response.status_code == 401
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
@pytest.mark.parametrize("depth", [1, 2, 3])
@pytest.mark.parametrize(
"reach,role",
[
["restricted", "editor"],
["restricted", "reader"],
["public", "reader"],
["authenticated", "reader"],
],
)
def test_api_documents_children_create_authenticated_forbidden(reach, role, depth):
"""
Authenticated users with no write access on a document should not be allowed
to create a nested document.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
for i in range(depth):
if i == 0:
document = factories.DocumentFactory(link_reach=reach, link_role=role)
else:
document = factories.DocumentFactory(parent=document, link_role="reader")
response = client.post(
f"/api/v1.0/documents/{document.id!s}/children/",
{
"title": "my document",
},
)
assert response.status_code == 403
assert Document.objects.count() == depth
@pytest.mark.parametrize("depth", [1, 2, 3])
@pytest.mark.parametrize(
"reach,role",
[
["public", "editor"],
["authenticated", "editor"],
],
)
def test_api_documents_children_create_authenticated_success(reach, role, depth):
"""
Authenticated users with write access on a document should be able
to create a nested document.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
for i in range(depth):
if i == 0:
document = factories.DocumentFactory(link_reach=reach, link_role=role)
else:
document = factories.DocumentFactory(parent=document, link_role="reader")
response = client.post(
f"/api/v1.0/documents/{document.id!s}/children/",
{
"title": "my child",
},
)
assert response.status_code == 201
child = Document.objects.get(id=response.json()["id"])
assert child.title == "my child"
assert child.link_reach == "restricted"
assert child.accesses.filter(role="owner", user=user).exists()
@pytest.mark.parametrize("depth", [1, 2, 3])
def test_api_documents_children_create_related_forbidden(depth):
"""
Authenticated users with a specific read access on a document should not be allowed
to create a nested document.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
for i in range(depth):
if i == 0:
document = factories.DocumentFactory(link_reach="restricted")
factories.UserDocumentAccessFactory(
user=user, document=document, role="reader"
)
else:
document = factories.DocumentFactory(
parent=document, link_reach="restricted"
)
response = client.post(
f"/api/v1.0/documents/{document.id!s}/children/",
{
"title": "my document",
},
)
assert response.status_code == 403
assert Document.objects.count() == depth
@pytest.mark.parametrize("depth", [1, 2, 3])
@pytest.mark.parametrize("role", ["editor", "administrator", "owner"])
def test_api_documents_children_create_related_success(role, depth):
"""
Authenticated users with a specific write access on a document should be
able to create a nested document.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
for i in range(depth):
if i == 0:
document = factories.DocumentFactory(link_reach="restricted")
factories.UserDocumentAccessFactory(user=user, document=document, role=role)
else:
document = factories.DocumentFactory(
parent=document, link_reach="restricted"
)
response = client.post(
f"/api/v1.0/documents/{document.id!s}/children/",
{
"title": "my child",
},
)
assert response.status_code == 201
child = Document.objects.get(id=response.json()["id"])
assert child.title == "my child"
assert child.link_reach == "restricted"
assert child.accesses.filter(role="owner", user=user).exists()
def test_api_documents_children_create_authenticated_title_null():
"""It should be possible to create several nested documents with a null title."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
parent = factories.DocumentFactory(
title=None, link_reach="authenticated", link_role="editor"
)
factories.DocumentFactory(title=None, parent=parent)
response = client.post(
f"/api/v1.0/documents/{parent.id!s}/children/", {}, format="json"
)
assert response.status_code == 201
assert Document.objects.filter(title__isnull=True).count() == 3
def test_api_documents_children_create_force_id_success():
"""It should be possible to force the document ID when creating a nested document."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
access = factories.UserDocumentAccessFactory(user=user, role="editor")
forced_id = uuid4()
response = client.post(
f"/api/v1.0/documents/{access.document.id!s}/children/",
{
"id": str(forced_id),
"title": "my document",
},
format="json",
)
assert response.status_code == 201
assert Document.objects.count() == 2
assert response.json()["id"] == str(forced_id)
def test_api_documents_children_create_force_id_existing():
"""
It should not be possible to use the ID of an existing document when forcing ID on creation.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
access = factories.UserDocumentAccessFactory(user=user, role="editor")
document = factories.DocumentFactory()
response = client.post(
f"/api/v1.0/documents/{access.document.id!s}/children/",
{
"id": str(document.id),
"title": "my document",
},
format="json",
)
assert response.status_code == 400
assert response.json() == {
"id": ["A document with this ID already exists. You cannot override it."]
}

View File

@@ -0,0 +1,555 @@
"""
Tests for Documents API endpoint in impress's core app: children list
"""
import random
from django.contrib.auth.models import AnonymousUser
import pytest
from rest_framework.test import APIClient
from core import factories
pytestmark = pytest.mark.django_db
def test_api_documents_children_list_anonymous_public_standalone():
"""Anonymous users should be allowed to retrieve the children of a public document."""
document = factories.DocumentFactory(link_reach="public")
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
factories.UserDocumentAccessFactory(document=child1)
response = APIClient().get(f"/api/v1.0/documents/{document.id!s}/children/")
assert response.status_code == 200
assert response.json() == {
"count": 2,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(AnonymousUser()),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 2,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 1,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
{
"abilities": child2.get_abilities(AnonymousUser()),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 2,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 0,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
],
}
def test_api_documents_children_list_anonymous_public_parent():
"""
Anonymous users should be allowed to retrieve the children of a document who
has a public ancestor.
"""
grand_parent = factories.DocumentFactory(link_reach="public")
parent = factories.DocumentFactory(
parent=grand_parent, link_reach=random.choice(["authenticated", "restricted"])
)
document = factories.DocumentFactory(
link_reach=random.choice(["authenticated", "restricted"]), parent=parent
)
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
factories.UserDocumentAccessFactory(document=child1)
response = APIClient().get(f"/api/v1.0/documents/{document.id!s}/children/")
assert response.status_code == 200
assert response.json() == {
"count": 2,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(AnonymousUser()),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 4,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 1,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
{
"abilities": child2.get_abilities(AnonymousUser()),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 4,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 0,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
],
}
@pytest.mark.parametrize("reach", ["restricted", "authenticated"])
def test_api_documents_children_list_anonymous_restricted_or_authenticated(reach):
"""
Anonymous users should not be able to retrieve children of a document that is not public.
"""
document = factories.DocumentFactory(link_reach=reach)
factories.DocumentFactory.create_batch(2, parent=document)
response = APIClient().get(f"/api/v1.0/documents/{document.id!s}/children/")
assert response.status_code == 401
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
@pytest.mark.parametrize("reach", ["public", "authenticated"])
def test_api_documents_children_list_authenticated_unrelated_public_or_authenticated(
reach,
):
"""
Authenticated users should be able to retrieve the children of a public/authenticated
document to which they are not related.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach=reach)
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
factories.UserDocumentAccessFactory(document=child1)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/children/",
)
assert response.status_code == 200
assert response.json() == {
"count": 2,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(user),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 2,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 1,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
{
"abilities": child2.get_abilities(user),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 2,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 0,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
],
}
@pytest.mark.parametrize("reach", ["public", "authenticated"])
def test_api_documents_children_list_authenticated_public_or_authenticated_parent(
reach,
):
"""
Authenticated users should be allowed to retrieve the children of a document who
has a public or authenticated ancestor.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
grand_parent = factories.DocumentFactory(link_reach=reach)
parent = factories.DocumentFactory(parent=grand_parent, link_reach="restricted")
document = factories.DocumentFactory(link_reach="restricted", parent=parent)
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
factories.UserDocumentAccessFactory(document=child1)
response = client.get(f"/api/v1.0/documents/{document.id!s}/children/")
assert response.status_code == 200
assert response.json() == {
"count": 2,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(user),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 4,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 1,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
{
"abilities": child2.get_abilities(user),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 4,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 0,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
],
}
def test_api_documents_children_list_authenticated_unrelated_restricted():
"""
Authenticated users should not be allowed to retrieve the children of a document that is
restricted and to which they are not related.
"""
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="restricted")
child1, _child2 = factories.DocumentFactory.create_batch(2, parent=document)
factories.UserDocumentAccessFactory(document=child1)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/children/",
)
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
def test_api_documents_children_list_authenticated_related_direct():
"""
Authenticated users should be allowed to retrieve the children of a document
to which they are directly related whatever the role.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory()
access = factories.UserDocumentAccessFactory(document=document, user=user)
factories.UserDocumentAccessFactory(document=document)
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
factories.UserDocumentAccessFactory(document=child1)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/children/",
)
assert response.status_code == 200
assert response.json() == {
"count": 2,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(user),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 2,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 0,
"nb_accesses_ancestors": 3,
"nb_accesses_direct": 1,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [access.role],
},
{
"abilities": child2.get_abilities(user),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 2,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 2,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [access.role],
},
],
}
def test_api_documents_children_list_authenticated_related_parent():
"""
Authenticated users should be allowed to retrieve the children of a document if they
are related to one of its ancestors whatever the role.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
grand_parent = factories.DocumentFactory(link_reach="restricted")
parent = factories.DocumentFactory(parent=grand_parent, link_reach="restricted")
document = factories.DocumentFactory(parent=parent, link_reach="restricted")
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
factories.UserDocumentAccessFactory(document=child1)
grand_parent_access = factories.UserDocumentAccessFactory(
document=grand_parent, user=user
)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/children/",
)
assert response.status_code == 200
assert response.json() == {
"count": 2,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(user),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 4,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 0,
"nb_accesses_ancestors": 2,
"nb_accesses_direct": 1,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [grand_parent_access.role],
},
{
"abilities": child2.get_abilities(user),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 4,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [grand_parent_access.role],
},
],
}
def test_api_documents_children_list_authenticated_related_child():
"""
Authenticated users should not be allowed to retrieve all the children of a document
as a result of being related to one of its children.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="restricted")
child1, _child2 = factories.DocumentFactory.create_batch(2, parent=document)
factories.UserDocumentAccessFactory(document=child1, user=user)
factories.UserDocumentAccessFactory(document=document)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/children/",
)
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
def test_api_documents_children_list_authenticated_related_team_none(mock_user_teams):
"""
Authenticated users should not be able to retrieve the children of a restricted document
related to teams in which the user is not.
"""
mock_user_teams.return_value = []
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="restricted")
factories.DocumentFactory.create_batch(2, parent=document)
factories.TeamDocumentAccessFactory(document=document, team="myteam")
response = client.get(f"/api/v1.0/documents/{document.id!s}/children/")
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
def test_api_documents_children_list_authenticated_related_team_members(
mock_user_teams,
):
"""
Authenticated users should be allowed to retrieve the children of a document to which they
are related via a team whatever the role.
"""
mock_user_teams.return_value = ["myteam"]
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="restricted")
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
access = factories.TeamDocumentAccessFactory(document=document, team="myteam")
response = client.get(f"/api/v1.0/documents/{document.id!s}/children/")
# pylint: disable=R0801
assert response.status_code == 200
assert response.json() == {
"count": 2,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(user),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 2,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 0,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [access.role],
},
{
"abilities": child2.get_abilities(user),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 2,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [access.role],
},
],
}

View File

@@ -0,0 +1,121 @@
"""Test on the CORS proxy API for documents."""
import pytest
import responses
from rest_framework.test import APIClient
from core import factories
pytestmark = pytest.mark.django_db
@responses.activate
def test_api_docs_cors_proxy_valid_url():
"""Test the CORS proxy API for documents with a valid URL."""
document = factories.DocumentFactory(link_reach="public")
client = APIClient()
url_to_fetch = "https://external-url.com/assets/logo-gouv.png"
responses.get(url_to_fetch, body=b"", status=200, content_type="image/png")
response = client.get(
f"/api/v1.0/documents/{document.id!s}/cors-proxy/?url={url_to_fetch}"
)
assert response.status_code == 200
assert response.headers["Content-Type"] == "image/png"
assert response.headers["Content-Disposition"] == "attachment;"
assert (
response.headers["Content-Security-Policy"]
== "default-src 'none'; img-src 'none' data:;"
)
assert response.streaming_content
def test_api_docs_cors_proxy_without_url_query_string():
"""Test the CORS proxy API for documents without a URL query string."""
document = factories.DocumentFactory(link_reach="public")
client = APIClient()
response = client.get(f"/api/v1.0/documents/{document.id!s}/cors-proxy/")
assert response.status_code == 400
assert response.json() == {"detail": "Missing 'url' query parameter"}
@responses.activate
def test_api_docs_cors_proxy_anonymous_document_not_public():
"""Test the CORS proxy API for documents with an anonymous user and a non-public document."""
document = factories.DocumentFactory(link_reach="authenticated")
client = APIClient()
url_to_fetch = "https://external-url.com/assets/logo-gouv.png"
responses.get(url_to_fetch, body=b"", status=200, content_type="image/png")
response = client.get(
f"/api/v1.0/documents/{document.id!s}/cors-proxy/?url={url_to_fetch}"
)
assert response.status_code == 401
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
@responses.activate
def test_api_docs_cors_proxy_authenticated_user_accessing_protected_doc():
"""
Test the CORS proxy API for documents with an authenticated user accessing a protected
document.
"""
document = factories.DocumentFactory(link_reach="authenticated")
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
url_to_fetch = "https://external-url.com/assets/logo-gouv.png"
responses.get(url_to_fetch, body=b"", status=200, content_type="image/png")
response = client.get(
f"/api/v1.0/documents/{document.id!s}/cors-proxy/?url={url_to_fetch}"
)
assert response.status_code == 200
assert response.headers["Content-Type"] == "image/png"
assert response.headers["Content-Disposition"] == "attachment;"
assert (
response.headers["Content-Security-Policy"]
== "default-src 'none'; img-src 'none' data:;"
)
assert response.streaming_content
@responses.activate
def test_api_docs_cors_proxy_authenticated_not_accessing_restricted_doc():
"""
Test the CORS proxy API for documents with an authenticated user not accessing a restricted
document.
"""
document = factories.DocumentFactory(link_reach="restricted")
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
url_to_fetch = "https://external-url.com/assets/logo-gouv.png"
responses.get(url_to_fetch, body=b"", status=200, content_type="image/png")
response = client.get(
f"/api/v1.0/documents/{document.id!s}/cors-proxy/?url={url_to_fetch}"
)
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
@responses.activate
def test_api_docs_cors_proxy_unsupported_media_type():
"""Test the CORS proxy API for documents with an unsupported media type."""
document = factories.DocumentFactory(link_reach="public")
client = APIClient()
url_to_fetch = "https://external-url.com/assets/index.html"
responses.get(url_to_fetch, body=b"", status=200, content_type="text/html")
response = client.get(
f"/api/v1.0/documents/{document.id!s}/cors-proxy/?url={url_to_fetch}"
)
assert response.status_code == 415

View File

@@ -77,6 +77,37 @@ def test_api_documents_delete_authenticated_not_owner(via, role, mock_user_teams
assert models.Document.objects.count() == 2
@pytest.mark.parametrize("depth", [1, 2, 3])
def test_api_documents_delete_authenticated_owner_of_ancestor(depth):
"""
Authenticated users should not be able to delete a document for which
they are only owner of an ancestor.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
documents = []
for i in range(depth):
documents.append(
factories.UserDocumentAccessFactory(role="owner", user=user).document
if i == 0
else factories.DocumentFactory(parent=documents[-1])
)
assert models.Document.objects.count() == depth
response = client.delete(
f"/api/v1.0/documents/{documents[-1].id}/",
)
assert response.status_code == 204
# Make sure it is only a soft delete
assert models.Document.objects.count() == depth
assert models.Document.objects.filter(deleted_at__isnull=True).count() == depth - 1
assert models.Document.objects.filter(deleted_at__isnull=False).count() == 1
@pytest.mark.parametrize("via", VIA)
def test_api_documents_delete_authenticated_owner(via, mock_user_teams):
"""
@@ -101,4 +132,8 @@ def test_api_documents_delete_authenticated_owner(via, mock_user_teams):
)
assert response.status_code == 204
assert models.Document.objects.exists() is False
# Make sure it is only a soft delete
assert models.Document.objects.count() == 1
assert models.Document.objects.filter(deleted_at__isnull=True).exists() is False
assert models.Document.objects.filter(deleted_at__isnull=False).count() == 1

View File

@@ -0,0 +1,696 @@
"""
Tests for Documents API endpoint in impress's core app: descendants
"""
import random
from django.contrib.auth.models import AnonymousUser
import pytest
from rest_framework.test import APIClient
from core import factories
pytestmark = pytest.mark.django_db
def test_api_documents_descendants_list_anonymous_public_standalone():
"""Anonymous users should be allowed to retrieve the descendants of a public document."""
document = factories.DocumentFactory(link_reach="public")
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
grand_child = factories.DocumentFactory(parent=child1)
factories.UserDocumentAccessFactory(document=child1)
response = APIClient().get(f"/api/v1.0/documents/{document.id!s}/descendants/")
assert response.status_code == 200
assert response.json() == {
"count": 3,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(AnonymousUser()),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 2,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 1,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 1,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
{
"abilities": grand_child.get_abilities(AnonymousUser()),
"created_at": grand_child.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(grand_child.creator.id),
"depth": 3,
"excerpt": grand_child.excerpt,
"id": str(grand_child.id),
"is_favorite": False,
"link_reach": grand_child.link_reach,
"link_role": grand_child.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 0,
"path": grand_child.path,
"title": grand_child.title,
"updated_at": grand_child.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
{
"abilities": child2.get_abilities(AnonymousUser()),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 2,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 0,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
],
}
def test_api_documents_descendants_list_anonymous_public_parent():
"""
Anonymous users should be allowed to retrieve the descendants of a document who
has a public ancestor.
"""
grand_parent = factories.DocumentFactory(link_reach="public")
parent = factories.DocumentFactory(
parent=grand_parent, link_reach=random.choice(["authenticated", "restricted"])
)
document = factories.DocumentFactory(
link_reach=random.choice(["authenticated", "restricted"]), parent=parent
)
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
grand_child = factories.DocumentFactory(parent=child1)
factories.UserDocumentAccessFactory(document=child1)
response = APIClient().get(f"/api/v1.0/documents/{document.id!s}/descendants/")
assert response.status_code == 200
assert response.json() == {
"count": 3,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(AnonymousUser()),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 4,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 1,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 1,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
{
"abilities": grand_child.get_abilities(AnonymousUser()),
"created_at": grand_child.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(grand_child.creator.id),
"depth": 5,
"excerpt": grand_child.excerpt,
"id": str(grand_child.id),
"is_favorite": False,
"link_reach": grand_child.link_reach,
"link_role": grand_child.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 0,
"path": grand_child.path,
"title": grand_child.title,
"updated_at": grand_child.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
{
"abilities": child2.get_abilities(AnonymousUser()),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 4,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 0,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
],
}
@pytest.mark.parametrize("reach", ["restricted", "authenticated"])
def test_api_documents_descendants_list_anonymous_restricted_or_authenticated(reach):
"""
Anonymous users should not be able to retrieve descendants of a document that is not public.
"""
document = factories.DocumentFactory(link_reach=reach)
child = factories.DocumentFactory(parent=document)
_grand_child = factories.DocumentFactory(parent=child)
response = APIClient().get(f"/api/v1.0/documents/{document.id!s}/descendants/")
assert response.status_code == 401
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
@pytest.mark.parametrize("reach", ["public", "authenticated"])
def test_api_documents_descendants_list_authenticated_unrelated_public_or_authenticated(
reach,
):
"""
Authenticated users should be able to retrieve the descendants of a public/authenticated
document to which they are not related.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach=reach)
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
grand_child = factories.DocumentFactory(parent=child1)
factories.UserDocumentAccessFactory(document=child1)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/descendants/",
)
assert response.status_code == 200
assert response.json() == {
"count": 3,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(user),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 2,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 1,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 1,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
{
"abilities": grand_child.get_abilities(user),
"created_at": grand_child.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(grand_child.creator.id),
"depth": 3,
"excerpt": grand_child.excerpt,
"id": str(grand_child.id),
"is_favorite": False,
"link_reach": grand_child.link_reach,
"link_role": grand_child.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 0,
"path": grand_child.path,
"title": grand_child.title,
"updated_at": grand_child.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
{
"abilities": child2.get_abilities(user),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 2,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 0,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
],
}
@pytest.mark.parametrize("reach", ["public", "authenticated"])
def test_api_documents_descendants_list_authenticated_public_or_authenticated_parent(
reach,
):
"""
Authenticated users should be allowed to retrieve the descendants of a document who
has a public or authenticated ancestor.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
grand_parent = factories.DocumentFactory(link_reach=reach)
parent = factories.DocumentFactory(parent=grand_parent, link_reach="restricted")
document = factories.DocumentFactory(link_reach="restricted", parent=parent)
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
grand_child = factories.DocumentFactory(parent=child1)
factories.UserDocumentAccessFactory(document=child1)
response = client.get(f"/api/v1.0/documents/{document.id!s}/descendants/")
assert response.status_code == 200
assert response.json() == {
"count": 3,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(user),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 4,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 1,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 1,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
{
"abilities": grand_child.get_abilities(user),
"created_at": grand_child.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(grand_child.creator.id),
"depth": 5,
"excerpt": grand_child.excerpt,
"id": str(grand_child.id),
"is_favorite": False,
"link_reach": grand_child.link_reach,
"link_role": grand_child.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 0,
"path": grand_child.path,
"title": grand_child.title,
"updated_at": grand_child.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
{
"abilities": child2.get_abilities(user),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 4,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 0,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
},
],
}
def test_api_documents_descendants_list_authenticated_unrelated_restricted():
"""
Authenticated users should not be allowed to retrieve the descendants of a document that is
restricted and to which they are not related.
"""
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="restricted")
child1, _child2 = factories.DocumentFactory.create_batch(2, parent=document)
_grand_child = factories.DocumentFactory(parent=child1)
factories.UserDocumentAccessFactory(document=child1)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/descendants/",
)
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
def test_api_documents_descendants_list_authenticated_related_direct():
"""
Authenticated users should be allowed to retrieve the descendants of a document
to which they are directly related whatever the role.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory()
access = factories.UserDocumentAccessFactory(document=document, user=user)
factories.UserDocumentAccessFactory(document=document)
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
factories.UserDocumentAccessFactory(document=child1)
grand_child = factories.DocumentFactory(parent=child1)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/descendants/",
)
assert response.status_code == 200
assert response.json() == {
"count": 3,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(user),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 2,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 1,
"nb_accesses_ancestors": 3,
"nb_accesses_direct": 1,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [access.role],
},
{
"abilities": grand_child.get_abilities(user),
"created_at": grand_child.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(grand_child.creator.id),
"depth": 3,
"excerpt": grand_child.excerpt,
"id": str(grand_child.id),
"is_favorite": False,
"link_reach": grand_child.link_reach,
"link_role": grand_child.link_role,
"numchild": 0,
"nb_accesses_ancestors": 3,
"nb_accesses_direct": 0,
"path": grand_child.path,
"title": grand_child.title,
"updated_at": grand_child.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [access.role],
},
{
"abilities": child2.get_abilities(user),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 2,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 2,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [access.role],
},
],
}
def test_api_documents_descendants_list_authenticated_related_parent():
"""
Authenticated users should be allowed to retrieve the descendants of a document if they
are related to one of its ancestors whatever the role.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
grand_parent = factories.DocumentFactory(link_reach="restricted")
grand_parent_access = factories.UserDocumentAccessFactory(
document=grand_parent, user=user
)
parent = factories.DocumentFactory(parent=grand_parent, link_reach="restricted")
document = factories.DocumentFactory(parent=parent, link_reach="restricted")
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
factories.UserDocumentAccessFactory(document=child1)
grand_child = factories.DocumentFactory(parent=child1)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/descendants/",
)
assert response.status_code == 200
assert response.json() == {
"count": 3,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(user),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 4,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 1,
"nb_accesses_ancestors": 2,
"nb_accesses_direct": 1,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [grand_parent_access.role],
},
{
"abilities": grand_child.get_abilities(user),
"created_at": grand_child.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(grand_child.creator.id),
"depth": 5,
"excerpt": grand_child.excerpt,
"id": str(grand_child.id),
"is_favorite": False,
"link_reach": grand_child.link_reach,
"link_role": grand_child.link_role,
"numchild": 0,
"nb_accesses_ancestors": 2,
"nb_accesses_direct": 0,
"path": grand_child.path,
"title": grand_child.title,
"updated_at": grand_child.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [grand_parent_access.role],
},
{
"abilities": child2.get_abilities(user),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 4,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [grand_parent_access.role],
},
],
}
def test_api_documents_descendants_list_authenticated_related_child():
"""
Authenticated users should not be allowed to retrieve all the descendants of a document
as a result of being related to one of its children.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="restricted")
child1, _child2 = factories.DocumentFactory.create_batch(2, parent=document)
_grand_child = factories.DocumentFactory(parent=child1)
factories.UserDocumentAccessFactory(document=child1, user=user)
factories.UserDocumentAccessFactory(document=document)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/descendants/",
)
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
def test_api_documents_descendants_list_authenticated_related_team_none(
mock_user_teams,
):
"""
Authenticated users should not be able to retrieve the descendants of a restricted document
related to teams in which the user is not.
"""
mock_user_teams.return_value = []
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="restricted")
factories.DocumentFactory.create_batch(2, parent=document)
factories.TeamDocumentAccessFactory(document=document, team="myteam")
response = client.get(f"/api/v1.0/documents/{document.id!s}/descendants/")
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
def test_api_documents_descendants_list_authenticated_related_team_members(
mock_user_teams,
):
"""
Authenticated users should be allowed to retrieve the descendants of a document to which they
are related via a team whatever the role.
"""
mock_user_teams.return_value = ["myteam"]
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="restricted")
child1, child2 = factories.DocumentFactory.create_batch(2, parent=document)
grand_child = factories.DocumentFactory(parent=child1)
access = factories.TeamDocumentAccessFactory(document=document, team="myteam")
response = client.get(f"/api/v1.0/documents/{document.id!s}/descendants/")
# pylint: disable=R0801
assert response.status_code == 200
assert response.json() == {
"count": 3,
"next": None,
"previous": None,
"results": [
{
"abilities": child1.get_abilities(user),
"created_at": child1.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child1.creator.id),
"depth": 2,
"excerpt": child1.excerpt,
"id": str(child1.id),
"is_favorite": False,
"link_reach": child1.link_reach,
"link_role": child1.link_role,
"numchild": 1,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 0,
"path": child1.path,
"title": child1.title,
"updated_at": child1.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [access.role],
},
{
"abilities": grand_child.get_abilities(user),
"created_at": grand_child.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(grand_child.creator.id),
"depth": 3,
"excerpt": grand_child.excerpt,
"id": str(grand_child.id),
"is_favorite": False,
"link_reach": grand_child.link_reach,
"link_role": grand_child.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 0,
"path": grand_child.path,
"title": grand_child.title,
"updated_at": grand_child.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [access.role],
},
{
"abilities": child2.get_abilities(user),
"created_at": child2.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(child2.creator.id),
"depth": 2,
"excerpt": child2.excerpt,
"id": str(child2.id),
"is_favorite": False,
"link_reach": child2.link_reach,
"link_role": child2.link_role,
"numchild": 0,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 0,
"path": child2.path,
"title": child2.title,
"updated_at": child2.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [access.role],
},
],
}

View File

@@ -0,0 +1,88 @@
"""
Tests for Documents API endpoint in impress's core app: list
"""
import pytest
from faker import Faker
from rest_framework.test import APIClient
from core import factories
fake = Faker()
pytestmark = pytest.mark.django_db
# Filters: unknown field
def test_api_documents_descendants_filter_unknown_field():
"""
Trying to filter by an unknown field should be ignored.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory()
document = factories.DocumentFactory(users=[user])
expected_ids = {
str(document.id)
for document in factories.DocumentFactory.create_batch(2, parent=document)
}
response = client.get(
f"/api/v1.0/documents/{document.id!s}/descendants/?unknown=true"
)
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 2
assert {result["id"] for result in results} == expected_ids
# Filters: title
@pytest.mark.parametrize(
"query,nb_results",
[
("Project Alpha", 1), # Exact match
("project", 2), # Partial match (case-insensitive)
("Guide", 1), # Word match within a title
("Special", 0), # No match (nonexistent keyword)
("2024", 2), # Match by numeric keyword
("", 5), # Empty string
],
)
def test_api_documents_descendants_filter_title(query, nb_results):
"""Authenticated users should be able to search documents by their title."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(users=[user])
# Create documents with predefined titles
titles = [
"Project Alpha Documentation",
"Project Beta Overview",
"User Guide",
"Financial Report 2024",
"Annual Review 2024",
]
for title in titles:
factories.DocumentFactory(title=title, parent=document)
# Perform the search query
response = client.get(
f"/api/v1.0/documents/{document.id!s}/descendants/?title={query:s}"
)
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == nb_results
# Ensure all results contain the query in their title
for result in results:
assert query.lower().strip() in result["title"].lower()

View File

@@ -0,0 +1,80 @@
"""Test for the document favorite_list endpoint."""
import pytest
from rest_framework.test import APIClient
from core import factories, models
pytestmark = pytest.mark.django_db
def test_api_document_favorite_list_anonymous():
"""Anonymous users should receive a 401 error."""
client = APIClient()
response = client.get("/api/v1.0/documents/favorite_list/")
assert response.status_code == 401
def test_api_document_favorite_list_authenticated_no_favorite():
"""Authenticated users should receive an empty list."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
response = client.get("/api/v1.0/documents/favorite_list/")
assert response.status_code == 200
assert response.json() == {
"count": 0,
"next": None,
"previous": None,
"results": [],
}
def test_api_document_favorite_list_authenticated_with_favorite():
"""Authenticated users with a favorite should receive the favorite."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# User don't have access to this document, let say it had access and this access has been
# removed. It should not be in the favorite list anymore.
factories.DocumentFactory(favorited_by=[user])
document = factories.UserDocumentAccessFactory(
user=user, role=models.RoleChoices.READER, document__favorited_by=[user]
).document
response = client.get("/api/v1.0/documents/favorite_list/")
assert response.status_code == 200
assert response.json() == {
"count": 1,
"next": None,
"previous": None,
"results": [
{
"abilities": document.get_abilities(user),
"created_at": document.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(document.creator.id),
"content": document.content,
"depth": document.depth,
"excerpt": document.excerpt,
"id": str(document.id),
"is_favorite": True,
"link_reach": document.link_reach,
"link_role": document.link_role,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 1,
"numchild": document.numchild,
"path": document.path,
"title": document.title,
"updated_at": document.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": ["reader"],
}
],
}

View File

@@ -2,10 +2,11 @@
Tests for Documents API endpoint in impress's core app: list
"""
import operator
import random
from datetime import timedelta
from unittest import mock
from urllib.parse import urlencode
from django.utils import timezone
import pytest
from faker import Faker
@@ -23,7 +24,7 @@ pytestmark = pytest.mark.django_db
def test_api_documents_list_anonymous(reach, role):
"""
Anonymous users should not be allowed to list documents whatever the
link reach and the role
link reach and link role
"""
factories.DocumentFactory(link_reach=reach, link_role=role)
@@ -37,16 +38,16 @@ def test_api_documents_list_anonymous(reach, role):
def test_api_documents_list_format():
"""Validate the format of documents as returned by the list view."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
other_users = factories.UserFactory.create_batch(3)
document = factories.DocumentFactory(
users=[user, *factories.UserFactory.create_batch(2)],
users=factories.UserFactory.create_batch(2),
favorited_by=[user, *other_users],
link_traces=other_users,
)
access = factories.UserDocumentAccessFactory(document=document, user=user)
response = client.get("/api/v1.0/documents/")
@@ -62,18 +63,24 @@ def test_api_documents_list_format():
assert results[0] == {
"id": str(document.id),
"abilities": document.get_abilities(user),
"content": document.content,
"created_at": document.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(document.creator.id),
"depth": 1,
"excerpt": document.excerpt,
"is_favorite": True,
"link_reach": document.link_reach,
"link_role": document.link_role,
"nb_accesses": 3,
"nb_accesses_ancestors": 3,
"nb_accesses_direct": 3,
"numchild": 0,
"path": document.path,
"title": document.title,
"updated_at": document.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [access.role],
}
# pylint: disable=too-many-locals
def test_api_documents_list_authenticated_direct(django_assert_num_queries):
"""
Authenticated users should be able to list documents they are a direct
@@ -81,11 +88,10 @@ def test_api_documents_list_authenticated_direct(django_assert_num_queries):
than restricted.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
documents = [
document1, document2 = [
access.document
for access in factories.UserDocumentAccessFactory.create_batch(2, user=user)
]
@@ -95,16 +101,64 @@ def test_api_documents_list_authenticated_direct(django_assert_num_queries):
for role in models.LinkRoleChoices:
factories.DocumentFactory(link_reach=reach, link_role=role)
expected_ids = {str(document.id) for document in documents}
# Children of visible documents should not get listed even with a specific access
factories.DocumentFactory(parent=document1)
with django_assert_num_queries(3):
child1_with_access = factories.DocumentFactory(parent=document1)
factories.UserDocumentAccessFactory(user=user, document=child1_with_access)
middle_document = factories.DocumentFactory(parent=document2)
child2_with_access = factories.DocumentFactory(parent=middle_document)
factories.UserDocumentAccessFactory(user=user, document=child2_with_access)
# Children of hidden documents should get listed when visible by the logged-in user
hidden_root = factories.DocumentFactory()
child3_with_access = factories.DocumentFactory(parent=hidden_root)
factories.UserDocumentAccessFactory(user=user, document=child3_with_access)
child4_with_access = factories.DocumentFactory(parent=hidden_root)
factories.UserDocumentAccessFactory(user=user, document=child4_with_access)
# Documents that are soft deleted and children of a soft deleted document should not be listed
soft_deleted_document = factories.DocumentFactory(users=[user])
child_of_soft_deleted_document = factories.DocumentFactory(
users=[user],
parent=soft_deleted_document,
)
factories.DocumentFactory(users=[user], parent=child_of_soft_deleted_document)
soft_deleted_document.soft_delete()
# Documents that are permanently deleted and children of a permanently deleted
# document should not be listed
permanently_deleted_document = factories.DocumentFactory(users=[user])
child_of_permanently_deleted_document = factories.DocumentFactory(
users=[user], parent=permanently_deleted_document
)
factories.DocumentFactory(
users=[user], parent=child_of_permanently_deleted_document
)
fourty_days_ago = timezone.now() - timedelta(days=40)
with mock.patch("django.utils.timezone.now", return_value=fourty_days_ago):
permanently_deleted_document.soft_delete()
expected_ids = {
str(document1.id),
str(document2.id),
str(child3_with_access.id),
str(child4_with_access.id),
}
with django_assert_num_queries(12):
response = client.get("/api/v1.0/documents/")
# nb_accesses should now be cached
with django_assert_num_queries(4):
response = client.get("/api/v1.0/documents/")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 2
results_id = {result["id"] for result in results}
assert expected_ids == results_id
results_ids = {result["id"] for result in results}
assert expected_ids == results_ids
def test_api_documents_list_authenticated_via_team(
@@ -132,7 +186,11 @@ def test_api_documents_list_authenticated_via_team(
expected_ids = {str(document.id) for document in documents_team1 + documents_team2}
with django_assert_num_queries(3):
with django_assert_num_queries(14):
response = client.get("/api/v1.0/documents/")
# nb_accesses should now be cached
with django_assert_num_queries(4):
response = client.get("/api/v1.0/documents/")
assert response.status_code == 200
@@ -161,10 +219,12 @@ def test_api_documents_list_authenticated_link_reach_restricted(
other_document = factories.DocumentFactory(link_reach="public")
models.LinkTrace.objects.create(document=other_document, user=user)
with django_assert_num_queries(3):
response = client.get(
"/api/v1.0/documents/",
)
with django_assert_num_queries(6):
response = client.get("/api/v1.0/documents/")
# nb_accesses should now be cached
with django_assert_num_queries(4):
response = client.get("/api/v1.0/documents/")
assert response.status_code == 200
results = response.json()["results"]
@@ -186,21 +246,37 @@ def test_api_documents_list_authenticated_link_reach_public_or_authenticated(
client = APIClient()
client.force_login(user)
documents = [
document1, document2 = [
factories.DocumentFactory(link_traces=[user], link_reach=reach)
for reach in models.LinkReachChoices
if reach != "restricted"
]
expected_ids = {str(document.id) for document in documents}
factories.DocumentFactory(
link_reach=random.choice(["public", "authenticated"]),
link_traces=[user],
parent=document1,
)
with django_assert_num_queries(3):
response = client.get(
"/api/v1.0/documents/",
)
hidden_document = factories.DocumentFactory(
link_reach=random.choice(["public", "authenticated"])
)
visible_child = factories.DocumentFactory(
link_traces=[user],
link_reach=random.choice(["public", "authenticated"]),
parent=hidden_document,
)
expected_ids = {str(document1.id), str(document2.id), str(visible_child.id)}
with django_assert_num_queries(10):
response = client.get("/api/v1.0/documents/")
# nb_accesses should now be cached
with django_assert_num_queries(4):
response = client.get("/api/v1.0/documents/")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 2
results_id = {result["id"] for result in results}
assert expected_ids == results_id
@@ -253,6 +329,35 @@ def test_api_documents_list_pagination(
assert document_ids == []
def test_api_documents_list_pagination_force_page_size():
"""Page size can be set via querystring."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document_ids = [
str(access.document_id)
for access in factories.UserDocumentAccessFactory.create_batch(3, user=user)
]
# Force page size
response = client.get(
"/api/v1.0/documents/?page_size=2",
)
assert response.status_code == 200
content = response.json()
assert content["count"] == 3
assert content["next"] == "http://testserver/api/v1.0/documents/?page=2&page_size=2"
assert content["previous"] is None
assert len(content["results"]) == 2
for item in content["results"]:
document_ids.remove(item["id"])
def test_api_documents_list_authenticated_distinct():
"""A document with several related users should only be listed once."""
user = factories.UserFactory()
@@ -287,7 +392,11 @@ def test_api_documents_list_favorites_no_extra_queries(django_assert_num_queries
factories.DocumentFactory.create_batch(2, users=[user])
url = "/api/v1.0/documents/"
with django_assert_num_queries(3):
with django_assert_num_queries(14):
response = client.get(url)
# nb_accesses should now be cached
with django_assert_num_queries(4):
response = client.get(url)
assert response.status_code == 200
@@ -300,7 +409,7 @@ def test_api_documents_list_favorites_no_extra_queries(django_assert_num_queries
for document in special_documents:
models.DocumentFavorite.objects.create(document=document, user=user)
with django_assert_num_queries(3):
with django_assert_num_queries(4):
response = client.get(url)
assert response.status_code == 200
@@ -314,361 +423,3 @@ def test_api_documents_list_favorites_no_extra_queries(django_assert_num_queries
assert result["is_favorite"] is True
else:
assert result["is_favorite"] is False
def test_api_documents_list_filter_and_access_rights():
"""Filtering on querystring parameters should respect access rights."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
other_user = factories.UserFactory()
def random_favorited_by():
return random.choice([[], [user], [other_user]])
# Documents that should be listed to this user
listed_documents = [
factories.DocumentFactory(
link_reach="public",
link_traces=[user],
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
),
factories.DocumentFactory(
link_reach="authenticated",
link_traces=[user],
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
),
factories.DocumentFactory(
link_reach="restricted",
users=[user],
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
),
]
listed_ids = [str(doc.id) for doc in listed_documents]
word_list = [word for doc in listed_documents for word in doc.title.split(" ")]
# Documents that should not be listed to this user
factories.DocumentFactory(
link_reach="public",
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
)
factories.DocumentFactory(
link_reach="authenticated",
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
)
factories.DocumentFactory(
link_reach="restricted",
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
)
factories.DocumentFactory(
link_reach="restricted",
link_traces=[user],
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
)
filters = {
"link_reach": random.choice([None, *models.LinkReachChoices.values]),
"title": random.choice([None, *word_list]),
"favorite": random.choice([None, True, False]),
"creator": random.choice([None, user, other_user]),
"ordering": random.choice(
[
None,
"created_at",
"-created_at",
"is_favorite",
"-is_favorite",
"nb_accesses",
"-nb_accesses",
"title",
"-title",
"updated_at",
"-updated_at",
]
),
}
query_params = {key: value for key, value in filters.items() if value is not None}
querystring = urlencode(query_params)
response = client.get(f"/api/v1.0/documents/?{querystring:s}")
assert response.status_code == 200
results = response.json()["results"]
# Ensure all documents in results respect expected access rights
for result in results:
assert result["id"] in listed_ids
# Filters: ordering
def test_api_documents_list_ordering_default():
"""Documents should be ordered by descending "updated_at" by default"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(5, users=[user])
response = client.get("/api/v1.0/documents/")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 5
# Check that results are sorted by descending "updated_at" as expected
for i in range(4):
assert operator.ge(results[i]["updated_at"], results[i + 1]["updated_at"])
def test_api_documents_list_ordering_by_fields():
"""It should be possible to order by several fields"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(5, users=[user])
for parameter in [
"created_at",
"-created_at",
"is_favorite",
"-is_favorite",
"nb_accesses",
"-nb_accesses",
"title",
"-title",
"updated_at",
"-updated_at",
]:
is_descending = parameter.startswith("-")
field = parameter.lstrip("-")
querystring = f"?ordering={parameter}"
response = client.get(f"/api/v1.0/documents/{querystring:s}")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 5
# Check that results are sorted by the field in querystring as expected
compare = operator.ge if is_descending else operator.le
for i in range(4):
assert compare(results[i][field], results[i + 1][field])
# Filters: is_creator_me
def test_api_documents_list_filter_is_creator_me_true():
"""
Authenticated users should be able to filter documents they created.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(3, users=[user], creator=user)
factories.DocumentFactory.create_batch(2, users=[user])
response = client.get("/api/v1.0/documents/?is_creator_me=true")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 3
# Ensure all results are created by the current user
for result in results:
assert result["creator"] == str(user.id)
def test_api_documents_list_filter_is_creator_me_false():
"""
Authenticated users should be able to filter documents created by others.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(3, users=[user], creator=user)
factories.DocumentFactory.create_batch(2, users=[user])
response = client.get("/api/v1.0/documents/?is_creator_me=false")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 2
# Ensure all results are created by other users
for result in results:
assert result["creator"] != str(user.id)
def test_api_documents_list_filter_is_creator_me_invalid():
"""Filtering with an invalid `is_creator_me` value should do nothing."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(3, users=[user], creator=user)
factories.DocumentFactory.create_batch(2, users=[user])
response = client.get("/api/v1.0/documents/?is_creator_me=invalid")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 5
# Filters: is_favorite
def test_api_documents_list_filter_is_favorite_true():
"""
Authenticated users should be able to filter documents they marked as favorite.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(3, users=[user], favorited_by=[user])
factories.DocumentFactory.create_batch(2, users=[user])
response = client.get("/api/v1.0/documents/?is_favorite=true")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 3
# Ensure all results are marked as favorite by the current user
for result in results:
assert result["is_favorite"] is True
def test_api_documents_list_filter_is_favorite_false():
"""
Authenticated users should be able to filter documents they didn't mark as favorite.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(3, users=[user], favorited_by=[user])
factories.DocumentFactory.create_batch(2, users=[user])
response = client.get("/api/v1.0/documents/?is_favorite=false")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 2
# Ensure all results are not marked as favorite by the current user
for result in results:
assert result["is_favorite"] is False
def test_api_documents_list_filter_is_favorite_invalid():
"""Filtering with an invalid `is_favorite` value should do nothing."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(3, users=[user], favorited_by=[user])
factories.DocumentFactory.create_batch(2, users=[user])
response = client.get("/api/v1.0/documents/?is_favorite=invalid")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 5
# Filters: link_reach
@pytest.mark.parametrize("reach", models.LinkReachChoices.values)
def test_api_documents_list_filter_link_reach(reach):
"""Authenticated users should be able to filter documents by link reach."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(5, users=[user])
response = client.get(f"/api/v1.0/documents/?link_reach={reach:s}")
assert response.status_code == 200
results = response.json()["results"]
# Ensure all results have the chosen link reach
for result in results:
assert result["link_reach"] == reach
def test_api_documents_list_filter_link_reach_invalid():
"""Filtering with an invalid `link_reach` value should raise an error."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(3, users=[user])
response = client.get("/api/v1.0/documents/?link_reach=invalid")
assert response.status_code == 400
assert response.json() == {
"link_reach": [
"Select a valid choice. invalid is not one of the available choices."
]
}
# Filters: title
@pytest.mark.parametrize(
"query,nb_results",
[
("Project Alpha", 1), # Exact match
("project", 2), # Partial match (case-insensitive)
("Guide", 1), # Word match within a title
("Special", 0), # No match (nonexistent keyword)
("2024", 2), # Match by numeric keyword
("", 5), # Empty string
],
)
def test_api_documents_list_filter_title(query, nb_results):
"""Authenticated users should be able to search documents by their title."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Create documents with predefined titles
titles = [
"Project Alpha Documentation",
"Project Beta Overview",
"User Guide",
"Financial Report 2024",
"Annual Review 2024",
]
for title in titles:
factories.DocumentFactory(title=title, users=[user])
# Perform the search query
response = client.get(f"/api/v1.0/documents/?title={query:s}")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == nb_results
# Ensure all results contain the query in their title
for result in results:
assert query.lower().strip() in result["title"].lower()

View File

@@ -0,0 +1,356 @@
"""
Tests for Documents API endpoint in impress's core app: list
"""
import operator
import random
from urllib.parse import urlencode
import pytest
from faker import Faker
from rest_framework.test import APIClient
from core import factories, models
fake = Faker()
pytestmark = pytest.mark.django_db
def test_api_documents_list_filter_and_access_rights():
"""Filtering on querystring parameters should respect access rights."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
other_user = factories.UserFactory()
def random_favorited_by():
return random.choice([[], [user], [other_user]])
# Documents that should be listed to this user
listed_documents = [
factories.DocumentFactory(
link_reach="public",
link_traces=[user],
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
),
factories.DocumentFactory(
link_reach="authenticated",
link_traces=[user],
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
),
factories.DocumentFactory(
link_reach="restricted",
users=[user],
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
),
]
listed_ids = [str(doc.id) for doc in listed_documents]
word_list = [word for doc in listed_documents for word in doc.title.split(" ")]
# Documents that should not be listed to this user
factories.DocumentFactory(
link_reach="public",
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
)
factories.DocumentFactory(
link_reach="authenticated",
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
)
factories.DocumentFactory(
link_reach="restricted",
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
)
factories.DocumentFactory(
link_reach="restricted",
link_traces=[user],
favorited_by=random_favorited_by(),
creator=random.choice([user, other_user]),
)
filters = {
"link_reach": random.choice([None, *models.LinkReachChoices.values]),
"title": random.choice([None, *word_list]),
"favorite": random.choice([None, True, False]),
"creator": random.choice([None, user, other_user]),
"ordering": random.choice(
[
None,
"created_at",
"-created_at",
"is_favorite",
"-is_favorite",
"title",
"-title",
"updated_at",
"-updated_at",
]
),
}
query_params = {key: value for key, value in filters.items() if value is not None}
querystring = urlencode(query_params)
response = client.get(f"/api/v1.0/documents/?{querystring:s}")
assert response.status_code == 200
results = response.json()["results"]
# Ensure all documents in results respect expected access rights
for result in results:
assert result["id"] in listed_ids
# Filters: ordering
def test_api_documents_list_ordering_default():
"""Documents should be ordered by descending "updated_at" by default"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(5, users=[user])
response = client.get("/api/v1.0/documents/")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 5
# Check that results are sorted by descending "updated_at" as expected
for i in range(4):
assert operator.ge(results[i]["updated_at"], results[i + 1]["updated_at"])
def test_api_documents_list_ordering_by_fields():
"""It should be possible to order by several fields"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(5, users=[user])
for parameter in [
"created_at",
"-created_at",
"is_favorite",
"-is_favorite",
"title",
"-title",
"updated_at",
"-updated_at",
]:
is_descending = parameter.startswith("-")
field = parameter.lstrip("-")
querystring = f"?ordering={parameter}"
response = client.get(f"/api/v1.0/documents/{querystring:s}")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 5
# Check that results are sorted by the field in querystring as expected
compare = operator.ge if is_descending else operator.le
for i in range(4):
assert compare(results[i][field], results[i + 1][field])
# Filters: unknown field
def test_api_documents_list_filter_unknown_field():
"""
Trying to filter by an unknown field should raise a 400 error.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory()
expected_ids = {
str(document.id)
for document in factories.DocumentFactory.create_batch(2, users=[user])
}
response = client.get("/api/v1.0/documents/?unknown=true")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 2
assert {result["id"] for result in results} == expected_ids
# Filters: is_creator_me
def test_api_documents_list_filter_is_creator_me_true():
"""
Authenticated users should be able to filter documents they created.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(3, users=[user], creator=user)
factories.DocumentFactory.create_batch(2, users=[user])
response = client.get("/api/v1.0/documents/?is_creator_me=true")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 3
# Ensure all results are created by the current user
for result in results:
assert result["creator"] == str(user.id)
def test_api_documents_list_filter_is_creator_me_false():
"""
Authenticated users should be able to filter documents created by others.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(3, users=[user], creator=user)
factories.DocumentFactory.create_batch(2, users=[user])
response = client.get("/api/v1.0/documents/?is_creator_me=false")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 2
# Ensure all results are created by other users
for result in results:
assert result["creator"] != str(user.id)
def test_api_documents_list_filter_is_creator_me_invalid():
"""Filtering with an invalid `is_creator_me` value should do nothing."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(3, users=[user], creator=user)
factories.DocumentFactory.create_batch(2, users=[user])
response = client.get("/api/v1.0/documents/?is_creator_me=invalid")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 5
# Filters: is_favorite
def test_api_documents_list_filter_is_favorite_true():
"""
Authenticated users should be able to filter documents they marked as favorite.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(3, users=[user], favorited_by=[user])
factories.DocumentFactory.create_batch(2, users=[user])
response = client.get("/api/v1.0/documents/?is_favorite=true")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 3
# Ensure all results are marked as favorite by the current user
for result in results:
assert result["is_favorite"] is True
def test_api_documents_list_filter_is_favorite_false():
"""
Authenticated users should be able to filter documents they didn't mark as favorite.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(3, users=[user], favorited_by=[user])
factories.DocumentFactory.create_batch(2, users=[user])
response = client.get("/api/v1.0/documents/?is_favorite=false")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 2
# Ensure all results are not marked as favorite by the current user
for result in results:
assert result["is_favorite"] is False
def test_api_documents_list_filter_is_favorite_invalid():
"""Filtering with an invalid `is_favorite` value should do nothing."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.DocumentFactory.create_batch(3, users=[user], favorited_by=[user])
factories.DocumentFactory.create_batch(2, users=[user])
response = client.get("/api/v1.0/documents/?is_favorite=invalid")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 5
# Filters: title
@pytest.mark.parametrize(
"query,nb_results",
[
("Project Alpha", 1), # Exact match
("project", 2), # Partial match (case-insensitive)
("Guide", 1), # Word match within a title
("Special", 0), # No match (nonexistent keyword)
("2024", 2), # Match by numeric keyword
("", 5), # Empty string
],
)
def test_api_documents_list_filter_title(query, nb_results):
"""Authenticated users should be able to search documents by their title."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Create documents with predefined titles
titles = [
"Project Alpha Documentation",
"Project Beta Overview",
"User Guide",
"Financial Report 2024",
"Annual Review 2024",
]
for title in titles:
parent = factories.DocumentFactory() if random.choice([True, False]) else None
factories.DocumentFactory(title=title, users=[user], parent=parent)
# Perform the search query
response = client.get(f"/api/v1.0/documents/?title={query:s}")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == nb_results
# Ensure all results contain the query in their title
for result in results:
assert query.lower().strip() in result["title"].lower()

View File

@@ -64,6 +64,30 @@ def test_api_documents_media_auth_anonymous_public():
assert response.content.decode("utf-8") == "my prose"
def test_api_documents_media_auth_extensions():
"""Files with extensions of any format should work."""
document = factories.DocumentFactory(link_reach="public")
extensions = [
"c",
"go",
"gif",
"mp4",
"woff2",
"appimage",
]
for ext in extensions:
filename = f"{uuid.uuid4()!s}.{ext:s}"
key = f"{document.pk!s}/attachments/{filename:s}"
original_url = f"http://localhost/media/{key:s}"
response = APIClient().get(
"/api/v1.0/documents/media-auth/", HTTP_X_ORIGINAL_URL=original_url
)
assert response.status_code == 200
@pytest.mark.parametrize("reach", ["authenticated", "restricted"])
def test_api_documents_media_auth_anonymous_authenticated_or_restricted(reach):
"""

View File

@@ -0,0 +1,339 @@
"""
Test moving documents within the document tree via an detail action API endpoint.
"""
import random
from uuid import uuid4
from django.utils import timezone
import pytest
from rest_framework.test import APIClient
from core import enums, factories, models
pytestmark = pytest.mark.django_db
def test_api_documents_move_anonymous_user():
"""Anonymous users should not be able to move documents."""
document = factories.DocumentFactory()
target = factories.DocumentFactory()
response = APIClient().post(
f"/api/v1.0/documents/{document.id!s}/move/",
data={"target_document_id": str(target.id)},
)
assert response.status_code == 401
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
@pytest.mark.parametrize("role", [None, "reader", "editor"])
def test_api_documents_move_authenticated_document_no_permission(role):
"""
Authenticated users should not be able to move documents with insufficient
permissions on the origin document.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory()
target = factories.UserDocumentAccessFactory(user=user, role="owner").document
if role:
factories.UserDocumentAccessFactory(document=document, user=user, role=role)
response = client.post(
f"/api/v1.0/documents/{document.id!s}/move/",
data={"target_document_id": str(target.id)},
)
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
def test_api_documents_move_invalid_target_string():
"""Test for moving a document to an invalid target as a random string."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.UserDocumentAccessFactory(user=user, role="owner").document
response = client.post(
f"/api/v1.0/documents/{document.id!s}/move/",
data={"target_document_id": "non-existent-id"},
)
assert response.status_code == 400
assert response.json() == {"target_document_id": ["Must be a valid UUID."]}
def test_api_documents_move_invalid_target_uuid():
"""Test for moving a document to an invalid target that looks like a UUID."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.UserDocumentAccessFactory(user=user, role="owner").document
response = client.post(
f"/api/v1.0/documents/{document.id!s}/move/",
data={"target_document_id": str(uuid4())},
)
assert response.status_code == 400
assert response.json() == {
"target_document_id": "Target parent document does not exist."
}
def test_api_documents_move_invalid_position():
"""Test moving a document to an invalid position."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.UserDocumentAccessFactory(user=user, role="owner").document
target = factories.UserDocumentAccessFactory(user=user, role="owner").document
response = client.post(
f"/api/v1.0/documents/{document.id!s}/move/",
data={
"target_document_id": str(target.id),
"position": "invalid-position",
},
)
assert response.status_code == 400
assert response.json() == {
"position": ['"invalid-position" is not a valid choice.']
}
@pytest.mark.parametrize("position", enums.MoveNodePositionChoices.values)
@pytest.mark.parametrize("target_parent_role", models.RoleChoices.values)
@pytest.mark.parametrize("target_role", models.RoleChoices.values)
def test_api_documents_move_authenticated_target_roles_mocked(
target_role, target_parent_role, position
):
"""
Authenticated users with insufficient permissions on the target document (or its
parent depending on the position chosen), should not be allowed to move documents.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
power_roles = ["administrator", "owner"]
document = factories.DocumentFactory(users=[(user, random.choice(power_roles))])
children = factories.DocumentFactory.create_batch(3, parent=document)
target_parent = factories.DocumentFactory(users=[(user, target_parent_role)])
sibling1, target, sibling2 = factories.DocumentFactory.create_batch(
3, parent=target_parent
)
models.DocumentAccess.objects.create(document=target, user=user, role=target_role)
target_children = factories.DocumentFactory.create_batch(2, parent=target)
response = client.post(
f"/api/v1.0/documents/{document.id!s}/move/",
data={"target_document_id": str(target.id), "position": position},
)
document.refresh_from_db()
if (
position in ["first-child", "last-child"]
and (target_role in power_roles or target_parent_role in power_roles)
) or (
position in ["first-sibling", "last-sibling", "left", "right"]
and target_parent_role in power_roles
):
assert response.status_code == 200
assert response.json() == {"message": "Document moved successfully."}
match position:
case "first-child":
assert list(target.get_children()) == [document, *target_children]
case "last-child":
assert list(target.get_children()) == [*target_children, document]
case "first-sibling":
assert list(target.get_siblings()) == [
document,
sibling1,
target,
sibling2,
]
case "last-sibling":
assert list(target.get_siblings()) == [
sibling1,
target,
sibling2,
document,
]
case "left":
assert list(target.get_siblings()) == [
sibling1,
document,
target,
sibling2,
]
case "right":
assert list(target.get_siblings()) == [
sibling1,
target,
document,
sibling2,
]
case _:
raise ValueError(f"Invalid position: {position}")
# Verify that the document's children have also been moved
assert list(document.get_children()) == children
else:
assert response.status_code == 400
assert (
"You do not have permission to move documents"
in response.json()["target_document_id"]
)
assert document.is_root() is True
def test_api_documents_move_authenticated_deleted_document():
"""
It should not be possible to move a deleted document or its descendants, even
for an owner.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(
users=[(user, "owner")], deleted_at=timezone.now()
)
child = factories.DocumentFactory(parent=document, users=[(user, "owner")])
target = factories.DocumentFactory(users=[(user, "owner")])
# Try moving the deleted document
response = client.post(
f"/api/v1.0/documents/{document.id!s}/move/",
data={"target_document_id": str(target.id)},
)
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
# Verify that the document has not moved
document.refresh_from_db()
assert document.is_root() is True
# Try moving the child of the deleted document
response = client.post(
f"/api/v1.0/documents/{child.id!s}/move/",
data={"target_document_id": str(target.id)},
)
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
# Verify that the child has not moved
child.refresh_from_db()
assert child.is_child_of(document) is True
@pytest.mark.parametrize(
"position",
enums.MoveNodePositionChoices.values,
)
def test_api_documents_move_authenticated_deleted_target_as_child(position):
"""
It should not be possible to move a document as a child of a deleted target
even for a owner.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(users=[(user, "owner")])
target = factories.DocumentFactory(
users=[(user, "owner")], deleted_at=timezone.now()
)
child = factories.DocumentFactory(parent=target, users=[(user, "owner")])
# Try moving the document to the deleted target
response = client.post(
f"/api/v1.0/documents/{document.id!s}/move/",
data={"target_document_id": str(target.id), "position": position},
)
assert response.status_code == 400
assert response.json() == {
"target_document_id": "Target parent document does not exist."
}
# Verify that the document has not moved
document.refresh_from_db()
assert document.is_root() is True
# Try moving the document to the child of the deleted target
response = client.post(
f"/api/v1.0/documents/{document.id!s}/move/",
data={"target_document_id": str(child.id), "position": position},
)
assert response.status_code == 400
assert response.json() == {
"target_document_id": "Target parent document does not exist."
}
# Verify that the document has not moved
document.refresh_from_db()
assert document.is_root() is True
@pytest.mark.parametrize(
"position",
["first-sibling", "last-sibling", "left", "right"],
)
def test_api_documents_move_authenticated_deleted_target_as_sibling(position):
"""
It should not be possible to move a document as a sibling of a deleted target document
if the user has no rigths on its parent.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(users=[(user, "owner")])
target_parent = factories.DocumentFactory(
users=[(user, "owner")], deleted_at=timezone.now()
)
target = factories.DocumentFactory(users=[(user, "owner")], parent=target_parent)
# Try moving the document as a sibling of the target
response = client.post(
f"/api/v1.0/documents/{document.id!s}/move/",
data={"target_document_id": str(target.id), "position": position},
)
assert response.status_code == 400
assert response.json() == {
"target_document_id": "Target parent document does not exist."
}
# Verify that the document has not moved
document.refresh_from_db()
assert document.is_root() is True

View File

@@ -0,0 +1,126 @@
"""
Test restoring documents after a soft delete via the detail action API endpoint.
"""
from datetime import timedelta
from django.utils import timezone
import pytest
from rest_framework.test import APIClient
from core import factories
pytestmark = pytest.mark.django_db
def test_api_documents_restore_anonymous_user():
"""Anonymous users should not be able to restore deleted documents."""
now = timezone.now() - timedelta(days=15)
document = factories.DocumentFactory(deleted_at=now)
response = APIClient().post(f"/api/v1.0/documents/{document.id!s}/restore/")
assert response.status_code == 404
assert response.json() == {"detail": "Not found."}
document.refresh_from_db()
assert document.deleted_at == now
assert document.ancestors_deleted_at == now
@pytest.mark.parametrize("role", [None, "reader", "editor", "administrator"])
def test_api_documents_restore_authenticated_no_permission(role):
"""
Authenticated users who are not owners of a deleted document should
not be allowed to restore it.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
now = timezone.now() - timedelta(days=15)
document = factories.DocumentFactory(
deleted_at=now, link_reach="public", link_role="editor"
)
if role:
factories.UserDocumentAccessFactory(document=document, user=user, role=role)
response = client.post(f"/api/v1.0/documents/{document.id!s}/restore/")
assert response.status_code == 404
assert response.json() == {"detail": "Not found."}
document.refresh_from_db()
assert document.deleted_at == now
assert document.ancestors_deleted_at == now
def test_api_documents_restore_authenticated_owner_success():
"""The owner of a deleted document should be able to restore it."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
now = timezone.now() - timedelta(days=15)
document = factories.DocumentFactory(deleted_at=now)
factories.UserDocumentAccessFactory(document=document, user=user, role="owner")
response = client.post(f"/api/v1.0/documents/{document.id!s}/restore/")
assert response.status_code == 200
assert response.json() == {"detail": "Document has been successfully restored."}
document.refresh_from_db()
assert document.deleted_at is None
assert document.ancestors_deleted_at is None
def test_api_documents_restore_authenticated_owner_ancestor_deleted():
"""
The restored document should still be marked as deleted if one of its
ancestors is soft deleted as well.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
grand_parent = factories.DocumentFactory()
parent = factories.DocumentFactory(parent=grand_parent)
document = factories.DocumentFactory(parent=parent)
factories.UserDocumentAccessFactory(document=document, user=user, role="owner")
document.soft_delete()
document_deleted_at = document.deleted_at
assert document_deleted_at is not None
grand_parent.soft_delete()
grand_parent_deleted_at = grand_parent.deleted_at
assert grand_parent_deleted_at is not None
response = client.post(f"/api/v1.0/documents/{document.id!s}/restore/")
assert response.status_code == 200
assert response.json() == {"detail": "Document has been successfully restored."}
document.refresh_from_db()
assert document.deleted_at is None
# document is still marked as deleted
assert document.ancestors_deleted_at == grand_parent_deleted_at
assert grand_parent_deleted_at > document_deleted_at
def test_api_documents_restore_authenticated_owner_expired():
"""It should not be possible to restore a document beyond the allowed time limit."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
now = timezone.now() - timedelta(days=40)
document = factories.DocumentFactory(deleted_at=now)
factories.UserDocumentAccessFactory(document=document, user=user, role="owner")
response = client.post(f"/api/v1.0/documents/{document.id!s}/restore/")
assert response.status_code == 404
assert response.json() == {"detail": "Not found."}

View File

@@ -2,16 +2,21 @@
Tests for Documents API endpoint in impress's core app: retrieve
"""
import random
from datetime import timedelta
from unittest import mock
from django.utils import timezone
import pytest
from rest_framework.test import APIClient
from core import factories, models
from core.api import serializers
pytestmark = pytest.mark.django_db
def test_api_documents_retrieve_anonymous_public():
def test_api_documents_retrieve_anonymous_public_standalone():
"""Anonymous users should be allowed to retrieve public documents."""
document = factories.DocumentFactory(link_reach="public")
@@ -23,18 +28,30 @@ def test_api_documents_retrieve_anonymous_public():
"abilities": {
"accesses_manage": False,
"accesses_view": False,
"ai_transform": document.link_role == "editor",
"ai_translate": document.link_role == "editor",
"ai_transform": False,
"ai_translate": False,
"attachment_upload": document.link_role == "editor",
"children_create": False,
"children_list": True,
"collaboration_auth": True,
"cors_proxy": True,
"descendants": True,
"destroy": False,
# Anonymous user can't favorite a document even with read access
"favorite": False,
"invite_owner": False,
"link_configuration": False,
"link_select_options": {
"authenticated": ["reader", "editor"],
"public": ["reader", "editor"],
"restricted": ["reader", "editor"],
},
"media_auth": True,
"move": False,
"partial_update": document.link_role == "editor",
"restore": False,
"retrieve": True,
"tree": True,
"update": document.link_role == "editor",
"versions_destroy": False,
"versions_list": False,
@@ -43,12 +60,97 @@ def test_api_documents_retrieve_anonymous_public():
"content": document.content,
"created_at": document.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(document.creator.id),
"depth": 1,
"excerpt": document.excerpt,
"is_favorite": False,
"link_reach": "public",
"link_role": document.link_role,
"nb_accesses": 0,
"nb_accesses_ancestors": 0,
"nb_accesses_direct": 0,
"numchild": 0,
"path": document.path,
"title": document.title,
"updated_at": document.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
}
def test_api_documents_retrieve_anonymous_public_parent():
"""Anonymous users should be allowed to retrieve a document who has a public ancestor."""
grand_parent = factories.DocumentFactory(link_reach="public")
parent = factories.DocumentFactory(
parent=grand_parent, link_reach=random.choice(["authenticated", "restricted"])
)
document = factories.DocumentFactory(
link_reach=random.choice(["authenticated", "restricted"]), parent=parent
)
response = APIClient().get(f"/api/v1.0/documents/{document.id!s}/")
assert response.status_code == 200
links = document.get_ancestors().values("link_reach", "link_role")
assert response.json() == {
"id": str(document.id),
"abilities": {
"accesses_manage": False,
"accesses_view": False,
"ai_transform": False,
"ai_translate": False,
"attachment_upload": grand_parent.link_role == "editor",
"children_create": False,
"children_list": True,
"collaboration_auth": True,
"descendants": True,
"cors_proxy": True,
"destroy": False,
# Anonymous user can't favorite a document even with read access
"favorite": False,
"invite_owner": False,
"link_configuration": False,
"link_select_options": models.LinkReachChoices.get_select_options(links),
"media_auth": True,
"move": False,
"partial_update": grand_parent.link_role == "editor",
"restore": False,
"retrieve": True,
"tree": True,
"update": grand_parent.link_role == "editor",
"versions_destroy": False,
"versions_list": False,
"versions_retrieve": False,
},
"content": document.content,
"created_at": document.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(document.creator.id),
"depth": 3,
"excerpt": document.excerpt,
"is_favorite": False,
"link_reach": document.link_reach,
"link_role": document.link_role,
"nb_accesses_ancestors": 0,
"nb_accesses_direct": 0,
"numchild": 0,
"path": document.path,
"title": document.title,
"updated_at": document.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
}
def test_api_documents_retrieve_anonymous_public_child():
"""
Anonymous users having access to a document should not gain access to a parent document.
"""
document = factories.DocumentFactory(
link_reach=random.choice(["authenticated", "restricted"])
)
factories.DocumentFactory(link_reach="public", parent=document)
response = APIClient().get(f"/api/v1.0/documents/{document.id!s}/")
assert response.status_code == 401
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
@@ -68,8 +170,8 @@ def test_api_documents_retrieve_anonymous_restricted_or_authenticated(reach):
@pytest.mark.parametrize("reach", ["public", "authenticated"])
def test_api_documents_retrieve_authenticated_unrelated_public_or_authenticated(reach):
"""
Authenticated users should be able to retrieve a public document to which they are
not related.
Authenticated users should be able to retrieve a public/authenticated document to
which they are not related.
"""
user = factories.UserFactory()
@@ -90,14 +192,26 @@ def test_api_documents_retrieve_authenticated_unrelated_public_or_authenticated(
"ai_transform": document.link_role == "editor",
"ai_translate": document.link_role == "editor",
"attachment_upload": document.link_role == "editor",
"children_create": document.link_role == "editor",
"children_list": True,
"collaboration_auth": True,
"descendants": True,
"cors_proxy": True,
"destroy": False,
"favorite": True,
"invite_owner": False,
"media_auth": True,
"link_configuration": False,
"link_select_options": {
"authenticated": ["reader", "editor"],
"public": ["reader", "editor"],
"restricted": ["reader", "editor"],
},
"media_auth": True,
"move": False,
"partial_update": document.link_role == "editor",
"restore": False,
"retrieve": True,
"tree": True,
"update": document.link_role == "editor",
"versions_destroy": False,
"versions_list": False,
@@ -106,18 +220,111 @@ def test_api_documents_retrieve_authenticated_unrelated_public_or_authenticated(
"content": document.content,
"created_at": document.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(document.creator.id),
"depth": 1,
"excerpt": document.excerpt,
"is_favorite": False,
"link_reach": reach,
"link_role": document.link_role,
"nb_accesses": 0,
"nb_accesses_ancestors": 0,
"nb_accesses_direct": 0,
"numchild": 0,
"path": document.path,
"title": document.title,
"updated_at": document.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
}
assert (
models.LinkTrace.objects.filter(document=document, user=user).exists() is True
)
@pytest.mark.parametrize("reach", ["public", "authenticated"])
def test_api_documents_retrieve_authenticated_public_or_authenticated_parent(reach):
"""
Authenticated users should be allowed to retrieve a document who has a public or
authenticated ancestor.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
grand_parent = factories.DocumentFactory(link_reach=reach)
parent = factories.DocumentFactory(parent=grand_parent, link_reach="restricted")
document = factories.DocumentFactory(link_reach="restricted", parent=parent)
response = client.get(f"/api/v1.0/documents/{document.id!s}/")
assert response.status_code == 200
links = document.get_ancestors().values("link_reach", "link_role")
assert response.json() == {
"id": str(document.id),
"abilities": {
"accesses_manage": False,
"accesses_view": False,
"ai_transform": grand_parent.link_role == "editor",
"ai_translate": grand_parent.link_role == "editor",
"attachment_upload": grand_parent.link_role == "editor",
"children_create": grand_parent.link_role == "editor",
"children_list": True,
"collaboration_auth": True,
"descendants": True,
"cors_proxy": True,
"destroy": False,
"favorite": True,
"invite_owner": False,
"link_configuration": False,
"link_select_options": models.LinkReachChoices.get_select_options(links),
"move": False,
"media_auth": True,
"partial_update": grand_parent.link_role == "editor",
"restore": False,
"retrieve": True,
"tree": True,
"update": grand_parent.link_role == "editor",
"versions_destroy": False,
"versions_list": False,
"versions_retrieve": False,
},
"content": document.content,
"created_at": document.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(document.creator.id),
"depth": 3,
"excerpt": document.excerpt,
"is_favorite": False,
"link_reach": document.link_reach,
"link_role": document.link_role,
"nb_accesses_ancestors": 0,
"nb_accesses_direct": 0,
"numchild": 0,
"path": document.path,
"title": document.title,
"updated_at": document.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [],
}
@pytest.mark.parametrize("reach", ["public", "authenticated"])
def test_api_documents_retrieve_authenticated_public_or_authenticated_child(reach):
"""
Authenticated users having access to a document should not gain access to a parent document.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="restricted")
factories.DocumentFactory(link_reach=reach, parent=document)
response = client.get(f"/api/v1.0/documents/{document.id!s}/")
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
@pytest.mark.parametrize("reach", ["public", "authenticated"])
def test_api_documents_retrieve_authenticated_trace_twice(reach):
"""
@@ -179,10 +386,8 @@ def test_api_documents_retrieve_authenticated_related_direct():
client.force_login(user)
document = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=document, user=user)
access2 = factories.UserDocumentAccessFactory(document=document)
serializers.UserSerializer(instance=user)
serializers.UserSerializer(instance=access2.user)
access = factories.UserDocumentAccessFactory(document=document, user=user)
factories.UserDocumentAccessFactory(document=document)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/",
@@ -194,12 +399,144 @@ def test_api_documents_retrieve_authenticated_related_direct():
"content": document.content,
"creator": str(document.creator.id),
"created_at": document.created_at.isoformat().replace("+00:00", "Z"),
"depth": 1,
"excerpt": document.excerpt,
"is_favorite": False,
"link_reach": document.link_reach,
"link_role": document.link_role,
"nb_accesses": 2,
"nb_accesses_ancestors": 2,
"nb_accesses_direct": 2,
"numchild": 0,
"path": document.path,
"title": document.title,
"updated_at": document.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [access.role],
}
def test_api_documents_retrieve_authenticated_related_parent():
"""
Authenticated users should be allowed to retrieve a document if they are related
to one of its ancestors whatever the role.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
grand_parent = factories.DocumentFactory(link_reach="restricted")
parent = factories.DocumentFactory(parent=grand_parent, link_reach="restricted")
document = factories.DocumentFactory(parent=parent, link_reach="restricted")
access = factories.UserDocumentAccessFactory(document=grand_parent, user=user)
factories.UserDocumentAccessFactory(document=grand_parent)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/",
)
assert response.status_code == 200
links = document.get_ancestors().values("link_reach", "link_role")
assert response.json() == {
"id": str(document.id),
"abilities": {
"accesses_manage": access.role in ["administrator", "owner"],
"accesses_view": True,
"ai_transform": access.role != "reader",
"ai_translate": access.role != "reader",
"attachment_upload": access.role != "reader",
"children_create": access.role != "reader",
"children_list": True,
"collaboration_auth": True,
"descendants": True,
"cors_proxy": True,
"destroy": access.role == "owner",
"favorite": True,
"invite_owner": access.role == "owner",
"link_configuration": access.role in ["administrator", "owner"],
"link_select_options": models.LinkReachChoices.get_select_options(links),
"media_auth": True,
"move": access.role in ["administrator", "owner"],
"partial_update": access.role != "reader",
"restore": access.role == "owner",
"retrieve": True,
"tree": True,
"update": access.role != "reader",
"versions_destroy": access.role in ["administrator", "owner"],
"versions_list": True,
"versions_retrieve": True,
},
"content": document.content,
"creator": str(document.creator.id),
"created_at": document.created_at.isoformat().replace("+00:00", "Z"),
"depth": 3,
"excerpt": document.excerpt,
"is_favorite": False,
"link_reach": "restricted",
"link_role": document.link_role,
"nb_accesses_ancestors": 2,
"nb_accesses_direct": 0,
"numchild": 0,
"path": document.path,
"title": document.title,
"updated_at": document.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": [access.role],
}
def test_api_documents_retrieve_authenticated_related_nb_accesses():
"""Validate computation of number of accesses."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
grand_parent = factories.DocumentFactory(link_reach="restricted")
parent = factories.DocumentFactory(parent=grand_parent, link_reach="restricted")
document = factories.DocumentFactory(parent=parent, link_reach="restricted")
factories.UserDocumentAccessFactory(document=grand_parent, user=user)
factories.UserDocumentAccessFactory(document=parent)
factories.UserDocumentAccessFactory(document=document)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/",
)
assert response.status_code == 200
assert response.json()["nb_accesses_ancestors"] == 3
assert response.json()["nb_accesses_direct"] == 1
factories.UserDocumentAccessFactory(document=grand_parent)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/",
)
assert response.status_code == 200
assert response.json()["nb_accesses_ancestors"] == 4
assert response.json()["nb_accesses_direct"] == 1
def test_api_documents_retrieve_authenticated_related_child():
"""
Authenticated users should not be allowed to retrieve a document as a result of being
related to one of its children.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="restricted")
child = factories.DocumentFactory(parent=document)
factories.UserDocumentAccessFactory(document=child, user=user)
factories.UserDocumentAccessFactory(document=document)
response = client.get(
f"/api/v1.0/documents/{document.id!s}/",
)
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
@@ -238,16 +575,16 @@ def test_api_documents_retrieve_authenticated_related_team_none(mock_user_teams)
@pytest.mark.parametrize(
"teams",
"teams,roles",
[
["readers"],
["unknown", "readers"],
["editors"],
["unknown", "editors"],
[["readers"], ["reader"]],
[["unknown", "readers"], ["reader"]],
[["editors"], ["editor"]],
[["unknown", "editors"], ["editor"]],
],
)
def test_api_documents_retrieve_authenticated_related_team_members(
teams, mock_user_teams
teams, roles, mock_user_teams
):
"""
Authenticated users should be allowed to retrieve a document to which they
@@ -256,12 +593,10 @@ def test_api_documents_retrieve_authenticated_related_team_members(
mock_user_teams.return_value = teams
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach="restricted")
factories.TeamDocumentAccessFactory(
document=document, team="readers", role="reader"
)
@@ -285,25 +620,31 @@ def test_api_documents_retrieve_authenticated_related_team_members(
"content": document.content,
"created_at": document.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(document.creator.id),
"depth": 1,
"excerpt": document.excerpt,
"is_favorite": False,
"link_reach": "restricted",
"link_role": document.link_role,
"nb_accesses": 5,
"nb_accesses_ancestors": 5,
"nb_accesses_direct": 5,
"numchild": 0,
"path": document.path,
"title": document.title,
"updated_at": document.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": roles,
}
@pytest.mark.parametrize(
"teams",
"teams,roles",
[
["administrators"],
["editors", "administrators"],
["unknown", "administrators"],
[["administrators"], ["administrator"]],
[["editors", "administrators"], ["administrator", "editor"]],
[["unknown", "administrators"], ["administrator"]],
],
)
def test_api_documents_retrieve_authenticated_related_team_administrators(
teams, mock_user_teams
teams, roles, mock_user_teams
):
"""
Authenticated users should be allowed to retrieve a document to which they
@@ -341,26 +682,32 @@ def test_api_documents_retrieve_authenticated_related_team_administrators(
"content": document.content,
"created_at": document.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(document.creator.id),
"depth": 1,
"excerpt": document.excerpt,
"is_favorite": False,
"link_reach": "restricted",
"link_role": document.link_role,
"nb_accesses": 5,
"nb_accesses_ancestors": 5,
"nb_accesses_direct": 5,
"numchild": 0,
"path": document.path,
"title": document.title,
"updated_at": document.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": roles,
}
@pytest.mark.parametrize(
"teams",
"teams,roles",
[
["owners"],
["owners", "administrators"],
["members", "administrators", "owners"],
["unknown", "owners"],
[["owners"], ["owner"]],
[["owners", "administrators"], ["owner", "administrator"]],
[["members", "administrators", "owners"], ["owner", "administrator"]],
[["unknown", "owners"], ["owner"]],
],
)
def test_api_documents_retrieve_authenticated_related_team_owners(
teams, mock_user_teams
teams, roles, mock_user_teams
):
"""
Authenticated users should be allowed to retrieve a restricted document to which
@@ -369,7 +716,6 @@ def test_api_documents_retrieve_authenticated_related_team_owners(
mock_user_teams.return_value = teams
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
@@ -398,10 +744,232 @@ def test_api_documents_retrieve_authenticated_related_team_owners(
"content": document.content,
"created_at": document.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(document.creator.id),
"depth": 1,
"excerpt": document.excerpt,
"is_favorite": False,
"link_reach": "restricted",
"link_role": document.link_role,
"nb_accesses": 5,
"nb_accesses_ancestors": 5,
"nb_accesses_direct": 5,
"numchild": 0,
"path": document.path,
"title": document.title,
"updated_at": document.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": roles,
}
def test_api_documents_retrieve_user_roles(django_assert_max_num_queries):
"""
Roles should be annotated on querysets taking into account all documents ancestors.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
grand_parent = factories.DocumentFactory(
users=factories.UserFactory.create_batch(2)
)
parent = factories.DocumentFactory(
parent=grand_parent, users=factories.UserFactory.create_batch(2)
)
document = factories.DocumentFactory(
parent=parent, users=factories.UserFactory.create_batch(2)
)
accesses = (
factories.UserDocumentAccessFactory(document=grand_parent, user=user),
factories.UserDocumentAccessFactory(document=parent, user=user),
factories.UserDocumentAccessFactory(document=document, user=user),
)
expected_roles = {access.role for access in accesses}
with django_assert_max_num_queries(12):
response = client.get(f"/api/v1.0/documents/{document.id!s}/")
assert response.status_code == 200
user_roles = response.json()["user_roles"]
assert set(user_roles) == expected_roles
def test_api_documents_retrieve_numqueries_with_link_trace(django_assert_num_queries):
"""If the link traced already exists, the number of queries should be minimal."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(users=[user], link_traces=[user])
with django_assert_num_queries(5):
response = client.get(f"/api/v1.0/documents/{document.id!s}/")
with django_assert_num_queries(3):
response = client.get(f"/api/v1.0/documents/{document.id!s}/")
assert response.status_code == 200
assert response.json()["id"] == str(document.id)
# Soft/permanent delete
@pytest.mark.parametrize("depth", [1, 2, 3])
@pytest.mark.parametrize("reach", models.LinkReachChoices.values)
def test_api_documents_retrieve_soft_deleted_anonymous(reach, depth):
"""
A soft/permanently deleted public document should not be accessible via its
detail endpoint for anonymous users, and should return a 404.
"""
documents = []
for i in range(depth):
documents.append(
factories.DocumentFactory(link_reach=reach)
if i == 0
else factories.DocumentFactory(parent=documents[-1])
)
assert models.Document.objects.count() == depth
response = APIClient().get(f"/api/v1.0/documents/{documents[-1].id!s}/")
assert response.status_code == 200 if reach == "public" else 401
# Delete any one of the documents...
deleted_document = random.choice(documents)
deleted_document.soft_delete()
response = APIClient().get(f"/api/v1.0/documents/{documents[-1].id!s}/")
assert response.status_code == 404
assert response.json() == {"detail": "Not found."}
fourty_days_ago = timezone.now() - timedelta(days=40)
deleted_document.deleted_at = fourty_days_ago
deleted_document.ancestors_deleted_at = fourty_days_ago
deleted_document.save()
response = APIClient().get(f"/api/v1.0/documents/{documents[-1].id!s}/")
assert response.status_code == 404
assert response.json() == {"detail": "Not found."}
@pytest.mark.parametrize("depth", [1, 2, 3])
@pytest.mark.parametrize("reach", models.LinkReachChoices.values)
def test_api_documents_retrieve_soft_deleted_authenticated(reach, depth):
"""
A soft/permanently deleted document should not be accessible via its detail endpoint for
authenticated users not related to the document.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
documents = []
for i in range(depth):
documents.append(
factories.DocumentFactory(link_reach=reach)
if i == 0
else factories.DocumentFactory(parent=documents[-1])
)
assert models.Document.objects.count() == depth
response = client.get(f"/api/v1.0/documents/{documents[-1].id!s}/")
assert response.status_code == 200 if reach in ["public", "authenticated"] else 403
# Delete any one of the documents...
deleted_document = random.choice(documents)
deleted_document.soft_delete()
response = client.get(f"/api/v1.0/documents/{documents[-1].id!s}/")
assert response.status_code == 404
assert response.json() == {"detail": "Not found."}
fourty_days_ago = timezone.now() - timedelta(days=40)
deleted_document.deleted_at = fourty_days_ago
deleted_document.ancestors_deleted_at = fourty_days_ago
deleted_document.save()
response = client.get(f"/api/v1.0/documents/{documents[-1].id!s}/")
assert response.status_code == 404
assert response.json() == {"detail": "Not found."}
@pytest.mark.parametrize("depth", [1, 2, 3])
@pytest.mark.parametrize("role", models.RoleChoices.values)
def test_api_documents_retrieve_soft_deleted_related(role, depth):
"""
A soft deleted document should only be accessible via its detail endpoint by
users with specific "owner" access rights.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
documents = []
for i in range(depth):
documents.append(
factories.UserDocumentAccessFactory(role=role, user=user).document
if i == 0
else factories.DocumentFactory(parent=documents[-1])
)
assert models.Document.objects.count() == depth
document = documents[-1]
response = client.get(f"/api/v1.0/documents/{document.id!s}/")
assert response.status_code == 200
# Delete any one of the documents
deleted_document = random.choice(documents)
deleted_document.soft_delete()
response = client.get(f"/api/v1.0/documents/{document.id!s}/")
if role == "owner":
assert response.status_code == 200
assert response.json()["id"] == str(document.id)
else:
assert response.status_code == 404
assert response.json() == {"detail": "Not found."}
@pytest.mark.parametrize("depth", [1, 2, 3])
@pytest.mark.parametrize("role", models.RoleChoices.values)
def test_api_documents_retrieve_permanently_deleted_related(role, depth):
"""
A permanently deleted document should not be accessible via its detail endpoint for
authenticated users with specific access rights whatever their role.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
documents = []
for i in range(depth):
documents.append(
factories.UserDocumentAccessFactory(role=role, user=user).document
if i == 0
else factories.DocumentFactory(parent=documents[-1])
)
assert models.Document.objects.count() == depth
document = documents[-1]
response = client.get(f"/api/v1.0/documents/{document.id!s}/")
assert response.status_code == 200
# Delete any one of the documents
deleted_document = random.choice(documents)
fourty_days_ago = timezone.now() - timedelta(days=40)
with mock.patch("django.utils.timezone.now", return_value=fourty_days_ago):
deleted_document.soft_delete()
response = client.get(f"/api/v1.0/documents/{document.id!s}/")
assert response.status_code == 404
assert response.json() == {"detail": "Not found."}

View File

@@ -0,0 +1,285 @@
"""
Tests for Documents API endpoint in impress's core app: list
"""
from datetime import timedelta
from unittest import mock
from django.utils import timezone
import pytest
from faker import Faker
from rest_framework.pagination import PageNumberPagination
from rest_framework.test import APIClient
from core import factories, models
fake = Faker()
pytestmark = pytest.mark.django_db
@pytest.mark.parametrize("role", models.LinkRoleChoices.values)
@pytest.mark.parametrize("reach", models.LinkReachChoices.values)
def test_api_documents_trashbin_anonymous(reach, role):
"""
Anonymous users should not be allowed to list documents from the trashbin
whatever the link reach and link role
"""
factories.DocumentFactory(
link_reach=reach, link_role=role, deleted_at=timezone.now()
)
response = APIClient().get("/api/v1.0/documents/trashbin/")
assert response.status_code == 200
assert response.json() == {
"count": 0,
"next": None,
"previous": None,
"results": [],
}
def test_api_documents_trashbin_format():
"""Validate the format of documents as returned by the trashbin view."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
other_users = factories.UserFactory.create_batch(3)
document = factories.DocumentFactory(
deleted_at=timezone.now(),
users=factories.UserFactory.create_batch(2),
favorited_by=[user, *other_users],
link_traces=other_users,
)
factories.UserDocumentAccessFactory(document=document, user=user, role="owner")
response = client.get("/api/v1.0/documents/trashbin/")
assert response.status_code == 200
content = response.json()
results = content.pop("results")
assert content == {
"count": 1,
"next": None,
"previous": None,
}
assert len(results) == 1
assert results[0] == {
"id": str(document.id),
"abilities": {
"accesses_manage": True,
"accesses_view": True,
"ai_transform": True,
"ai_translate": True,
"attachment_upload": True,
"children_create": True,
"children_list": True,
"collaboration_auth": True,
"descendants": True,
"cors_proxy": True,
"destroy": True,
"favorite": True,
"invite_owner": True,
"link_configuration": True,
"link_select_options": {
"authenticated": ["reader", "editor"],
"public": ["reader", "editor"],
"restricted": ["reader", "editor"],
},
"media_auth": True,
"move": False, # Can't move a deleted document
"partial_update": True,
"restore": True,
"retrieve": True,
"tree": True,
"update": True,
"versions_destroy": True,
"versions_list": True,
"versions_retrieve": True,
},
"created_at": document.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(document.creator.id),
"depth": 1,
"excerpt": document.excerpt,
"link_reach": document.link_reach,
"link_role": document.link_role,
"nb_accesses_ancestors": 0,
"nb_accesses_direct": 3,
"numchild": 0,
"path": document.path,
"title": document.title,
"updated_at": document.updated_at.isoformat().replace("+00:00", "Z"),
"user_roles": ["owner"],
}
def test_api_documents_trashbin_authenticated_direct(django_assert_num_queries):
"""
The trashbin should only list deleted documents for which the current user is owner.
"""
now = timezone.now()
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document1, document2 = factories.DocumentFactory.create_batch(2, deleted_at=now)
models.DocumentAccess.objects.create(document=document1, user=user, role="owner")
models.DocumentAccess.objects.create(document=document2, user=user, role="owner")
# Unrelated documents
for reach in models.LinkReachChoices:
for role in models.LinkRoleChoices:
factories.DocumentFactory(link_reach=reach, link_role=role, deleted_at=now)
# Role other than "owner"
for role in models.RoleChoices.values:
if role == "owner":
continue
document_not_owner = factories.DocumentFactory(deleted_at=now)
models.DocumentAccess.objects.create(
document=document_not_owner, user=user, role=role
)
# Nested documents should also get listed
parent = factories.DocumentFactory(parent=document1)
document3 = factories.DocumentFactory(parent=parent, deleted_at=now)
models.DocumentAccess.objects.create(document=parent, user=user, role="owner")
# Permanently deleted documents should not be listed
fourty_days_ago = timezone.now() - timedelta(days=40)
permanently_deleted_document = factories.DocumentFactory(users=[(user, "owner")])
with mock.patch("django.utils.timezone.now", return_value=fourty_days_ago):
permanently_deleted_document.soft_delete()
expected_ids = {str(document1.id), str(document2.id), str(document3.id)}
with django_assert_num_queries(10):
response = client.get("/api/v1.0/documents/trashbin/")
with django_assert_num_queries(4):
response = client.get("/api/v1.0/documents/trashbin/")
assert response.status_code == 200
results = response.json()["results"]
results_ids = {result["id"] for result in results}
assert len(results) == 3
assert expected_ids == results_ids
def test_api_documents_trashbin_authenticated_via_team(
django_assert_num_queries, mock_user_teams
):
"""
Authenticated users should be able to list trashbin documents they own via a team.
"""
now = timezone.now()
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
mock_user_teams.return_value = ["team1", "team2", "unknown"]
deleted_document_team1 = factories.DocumentFactory(
teams=[("team1", "owner")], deleted_at=now
)
factories.DocumentFactory(teams=[("team1", "owner")])
factories.DocumentFactory(teams=[("team1", "administrator")], deleted_at=now)
factories.DocumentFactory(teams=[("team1", "administrator")])
deleted_document_team2 = factories.DocumentFactory(
teams=[("team2", "owner")], deleted_at=now
)
factories.DocumentFactory(teams=[("team2", "owner")])
factories.DocumentFactory(teams=[("team2", "administrator")], deleted_at=now)
factories.DocumentFactory(teams=[("team2", "administrator")])
expected_ids = {str(deleted_document_team1.id), str(deleted_document_team2.id)}
with django_assert_num_queries(7):
response = client.get("/api/v1.0/documents/trashbin/")
with django_assert_num_queries(3):
response = client.get("/api/v1.0/documents/trashbin/")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 2
results_id = {result["id"] for result in results}
assert expected_ids == results_id
@mock.patch.object(PageNumberPagination, "get_page_size", return_value=2)
def test_api_documents_trashbin_pagination(
_mock_page_size,
):
"""Pagination should work as expected."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document_ids = [
str(document.id)
for document in factories.DocumentFactory.create_batch(
3, deleted_at=timezone.now()
)
]
for document_id in document_ids:
models.DocumentAccess.objects.create(
document_id=document_id, user=user, role="owner"
)
# Get page 1
response = client.get("/api/v1.0/documents/trashbin/")
assert response.status_code == 200
content = response.json()
assert content["count"] == 3
assert content["next"] == "http://testserver/api/v1.0/documents/trashbin/?page=2"
assert content["previous"] is None
assert len(content["results"]) == 2
for item in content["results"]:
document_ids.remove(item["id"])
# Get page 2
response = client.get(
"/api/v1.0/documents/trashbin/?page=2",
)
assert response.status_code == 200
content = response.json()
assert content["count"] == 3
assert content["next"] is None
assert content["previous"] == "http://testserver/api/v1.0/documents/trashbin/"
assert len(content["results"]) == 1
document_ids.remove(content["results"][0]["id"])
assert document_ids == []
def test_api_documents_trashbin_distinct():
"""A document with several related users should only be listed once."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
other_user = factories.UserFactory()
document = factories.DocumentFactory(
users=[(user, "owner"), other_user], deleted_at=timezone.now()
)
response = client.get(
"/api/v1.0/documents/trashbin/",
)
assert response.status_code == 200
content = response.json()
assert len(content["results"]) == 1
assert content["results"][0]["id"] == str(document.id)

File diff suppressed because it is too large Load Diff

View File

@@ -16,6 +16,7 @@ from core.tests.conftest import TEAM, USER, VIA
pytestmark = pytest.mark.django_db
@pytest.mark.parametrize("via_parent", [True, False])
@pytest.mark.parametrize(
"reach, role",
[
@@ -26,12 +27,18 @@ pytestmark = pytest.mark.django_db
("public", "reader"),
],
)
def test_api_documents_update_anonymous_forbidden(reach, role):
def test_api_documents_update_anonymous_forbidden(reach, role, via_parent):
"""
Anonymous users should not be allowed to update a document when link
configuration does not allow it.
"""
document = factories.DocumentFactory(link_reach=reach, link_role=role)
if via_parent:
grand_parent = factories.DocumentFactory(link_reach=reach, link_role=role)
parent = factories.DocumentFactory(parent=grand_parent, link_reach="restricted")
document = factories.DocumentFactory(parent=parent, link_reach="restricted")
else:
document = factories.DocumentFactory(link_reach=reach, link_role=role)
old_document_values = serializers.DocumentSerializer(instance=document).data
new_document_values = serializers.DocumentSerializer(
@@ -52,6 +59,7 @@ def test_api_documents_update_anonymous_forbidden(reach, role):
assert document_values == old_document_values
@pytest.mark.parametrize("via_parent", [True, False])
@pytest.mark.parametrize(
"reach,role",
[
@@ -61,7 +69,9 @@ def test_api_documents_update_anonymous_forbidden(reach, role):
("restricted", "editor"),
],
)
def test_api_documents_update_authenticated_unrelated_forbidden(reach, role):
def test_api_documents_update_authenticated_unrelated_forbidden(
reach, role, via_parent
):
"""
Authenticated users should not be allowed to update a document to which
they are not related if the link configuration does not allow it.
@@ -71,7 +81,12 @@ def test_api_documents_update_authenticated_unrelated_forbidden(reach, role):
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_reach=reach, link_role=role)
if via_parent:
grand_parent = factories.DocumentFactory(link_reach=reach, link_role=role)
parent = factories.DocumentFactory(parent=grand_parent, link_reach="restricted")
document = factories.DocumentFactory(parent=parent, link_reach="restricted")
else:
document = factories.DocumentFactory(link_reach=reach, link_role=role)
old_document_values = serializers.DocumentSerializer(instance=document).data
new_document_values = serializers.DocumentSerializer(
@@ -93,6 +108,7 @@ def test_api_documents_update_authenticated_unrelated_forbidden(reach, role):
assert document_values == old_document_values
@pytest.mark.parametrize("via_parent", [True, False])
@pytest.mark.parametrize(
"is_authenticated,reach,role",
[
@@ -102,10 +118,10 @@ def test_api_documents_update_authenticated_unrelated_forbidden(reach, role):
],
)
def test_api_documents_update_anonymous_or_authenticated_unrelated(
is_authenticated, reach, role
is_authenticated, reach, role, via_parent
):
"""
Authenticated users should be able to update a document to which
Anonymous and authenticated users should be able to update a document to which
they are not related if the link configuration allows it.
"""
client = APIClient()
@@ -116,7 +132,12 @@ def test_api_documents_update_anonymous_or_authenticated_unrelated(
else:
user = AnonymousUser()
document = factories.DocumentFactory(link_reach=reach, link_role=role)
if via_parent:
grand_parent = factories.DocumentFactory(link_reach=reach, link_role=role)
parent = factories.DocumentFactory(parent=grand_parent, link_reach="restricted")
document = factories.DocumentFactory(parent=parent, link_reach="restricted")
else:
document = factories.DocumentFactory(link_reach=reach, link_role=role)
old_document_values = serializers.DocumentSerializer(instance=document).data
new_document_values = serializers.DocumentSerializer(
@@ -137,8 +158,11 @@ def test_api_documents_update_anonymous_or_authenticated_unrelated(
"accesses",
"created_at",
"creator",
"depth",
"link_reach",
"link_role",
"numchild",
"path",
]:
assert value == old_document_values[key]
elif key == "updated_at":
@@ -147,24 +171,34 @@ def test_api_documents_update_anonymous_or_authenticated_unrelated(
assert value == new_document_values[key]
@pytest.mark.parametrize("via_parent", [True, False])
@pytest.mark.parametrize("via", VIA)
def test_api_documents_update_authenticated_reader(via, mock_user_teams):
def test_api_documents_update_authenticated_reader(via, via_parent, mock_user_teams):
"""
Users who are reader of a document but not administrators should
not be allowed to update it.
Users who are reader of a document should not be allowed to update it.
"""
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory(link_role="reader")
if via_parent:
grand_parent = factories.DocumentFactory(link_reach="restricted")
parent = factories.DocumentFactory(parent=grand_parent, link_reach="restricted")
document = factories.DocumentFactory(parent=parent, link_reach="restricted")
access_document = grand_parent
else:
document = factories.DocumentFactory(link_reach="restricted")
access_document = document
if via == USER:
factories.UserDocumentAccessFactory(document=document, user=user, role="reader")
factories.UserDocumentAccessFactory(
document=access_document, user=user, role="reader"
)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamDocumentAccessFactory(
document=document, team="lasuite", role="reader"
document=access_document, team="lasuite", role="reader"
)
old_document_values = serializers.DocumentSerializer(instance=document).data
@@ -188,10 +222,11 @@ def test_api_documents_update_authenticated_reader(via, mock_user_teams):
assert document_values == old_document_values
@pytest.mark.parametrize("via_parent", [True, False])
@pytest.mark.parametrize("role", ["editor", "administrator", "owner"])
@pytest.mark.parametrize("via", VIA)
def test_api_documents_update_authenticated_editor_administrator_or_owner(
via, role, mock_user_teams
via, role, via_parent, mock_user_teams
):
"""A user who is editor, administrator or owner of a document should be allowed to update it."""
user = factories.UserFactory(with_owned_document=True)
@@ -199,13 +234,23 @@ def test_api_documents_update_authenticated_editor_administrator_or_owner(
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory()
if via_parent:
grand_parent = factories.DocumentFactory(link_reach="restricted")
parent = factories.DocumentFactory(parent=grand_parent, link_reach="restricted")
document = factories.DocumentFactory(parent=parent, link_reach="restricted")
access_document = grand_parent
else:
document = factories.DocumentFactory(link_reach="restricted")
access_document = document
if via == USER:
factories.UserDocumentAccessFactory(document=document, user=user, role=role)
factories.UserDocumentAccessFactory(
document=access_document, user=user, role=role
)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamDocumentAccessFactory(
document=document, team="lasuite", role=role
document=access_document, team="lasuite", role=role
)
old_document_values = serializers.DocumentSerializer(instance=document).data
@@ -227,55 +272,13 @@ def test_api_documents_update_authenticated_editor_administrator_or_owner(
"id",
"created_at",
"creator",
"depth",
"link_reach",
"link_role",
"nb_accesses",
]:
assert value == old_document_values[key]
elif key == "updated_at":
assert value > old_document_values[key]
else:
assert value == new_document_values[key]
@pytest.mark.parametrize("via", VIA)
def test_api_documents_update_authenticated_owners(via, mock_user_teams):
"""Administrators of a document should be allowed to update it."""
user = factories.UserFactory(with_owned_document=True)
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory()
if via == USER:
factories.UserDocumentAccessFactory(document=document, user=user, role="owner")
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
factories.TeamDocumentAccessFactory(
document=document, team="lasuite", role="owner"
)
old_document_values = serializers.DocumentSerializer(instance=document).data
new_document_values = serializers.DocumentSerializer(
instance=factories.DocumentFactory()
).data
response = client.put(
f"/api/v1.0/documents/{document.id!s}/", new_document_values, format="json"
)
assert response.status_code == 200
document = models.Document.objects.get(pk=document.pk)
document_values = serializers.DocumentSerializer(instance=document).data
for key, value in document_values.items():
if key in [
"id",
"created_at",
"creator",
"link_reach",
"link_role",
"nb_accesses",
"nb_accesses_ancestors",
"nb_accesses_direct",
"numchild",
"path",
]:
assert value == old_document_values[key]
elif key == "updated_at":

View File

@@ -0,0 +1,35 @@
import pytest
from core import factories
@pytest.mark.django_db
def test_update_blank_title_migration(migrator):
"""
Test that the migration fixes the titles of documents that are
"Untitled document", "Unbenanntes Dokument" or "Document sans titre"
"""
migrator.apply_initial_migration(("core", "0017_add_fields_for_soft_delete"))
english_doc = factories.DocumentFactory(title="Untitled document")
german_doc = factories.DocumentFactory(title="Unbenanntes Dokument")
french_doc = factories.DocumentFactory(title="Document sans titre")
other_doc = factories.DocumentFactory(title="My document")
assert english_doc.title == "Untitled document"
assert german_doc.title == "Unbenanntes Dokument"
assert french_doc.title == "Document sans titre"
assert other_doc.title == "My document"
# Apply the migration
migrator.apply_tested_migration(("core", "0018_update_blank_title"))
english_doc.refresh_from_db()
german_doc.refresh_from_db()
french_doc.refresh_from_db()
other_doc.refresh_from_db()
assert english_doc.title == None
assert german_doc.title == None
assert french_doc.title == None
assert other_doc.title == "My document"

View File

@@ -33,7 +33,7 @@ def test_openapi_client_schema():
)
assert output.getvalue() == ""
response = Client().get("/v1.0/swagger.json")
response = Client().get("/api/v1.0/swagger.json")
assert response.status_code == 200
with open(

View File

@@ -73,14 +73,14 @@ def test_api_template_accesses_list_authenticated_related(via, mock_user_teams):
user_access = models.TemplateAccess.objects.create(
template=template,
user=user,
role=random.choice(models.RoleChoices.choices)[0],
role=random.choice(models.RoleChoices.values),
)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
user_access = models.TemplateAccess.objects.create(
template=template,
team="lasuite",
role=random.choice(models.RoleChoices.choices)[0],
role=random.choice(models.RoleChoices.values),
)
access1 = factories.TeamTemplateAccessFactory(template=template)
@@ -219,7 +219,7 @@ def test_api_template_accesses_update_anonymous():
new_values = {
"id": uuid4(),
"user": factories.UserFactory().id,
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
api_client = APIClient()
@@ -252,7 +252,7 @@ def test_api_template_accesses_update_authenticated_unrelated():
new_values = {
"id": uuid4(),
"user": factories.UserFactory().id,
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
for field, value in new_values.items():
@@ -294,7 +294,7 @@ def test_api_template_accesses_update_authenticated_editor_or_reader(
new_values = {
"id": uuid4(),
"user": factories.UserFactory().id,
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
for field, value in new_values.items():
@@ -398,7 +398,7 @@ def test_api_template_accesses_update_administrator_from_owner(via, mock_user_te
new_values = {
"id": uuid4(),
"user_id": factories.UserFactory().id,
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
for field, value in new_values.items():
@@ -497,7 +497,7 @@ def test_api_template_accesses_update_owner(via, mock_user_teams):
new_values = {
"id": uuid4(),
"user_id": factories.UserFactory().id,
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
}
for field, value in new_values.items():

View File

@@ -23,7 +23,7 @@ def test_api_template_accesses_create_anonymous():
{
"user": str(other_user.id),
"template": str(template.id),
"role": random.choice(models.RoleChoices.choices)[0],
"role": random.choice(models.RoleChoices.values),
},
format="json",
)

View File

@@ -1,208 +0,0 @@
"""
Test users API endpoints in the impress core app.
"""
import pytest
from rest_framework.test import APIClient
from core import factories
from core.tests.conftest import TEAM, USER, VIA
pytestmark = pytest.mark.django_db
def test_api_templates_generate_document_anonymous_public():
"""Anonymous users can generate pdf document with public templates."""
template = factories.TemplateFactory(is_public=True)
data = {
"body": "# Test markdown body",
}
response = APIClient().post(
f"/api/v1.0/templates/{template.id!s}/generate-document/",
data,
format="json",
)
assert response.status_code == 200
assert response.headers["content-type"] == "application/pdf"
def test_api_templates_generate_document_anonymous_not_public():
"""
Anonymous users should not be allowed to generate pdf document with templates
that are not marked as public.
"""
template = factories.TemplateFactory(is_public=False)
data = {
"body": "# Test markdown body",
}
response = APIClient().post(
f"/api/v1.0/templates/{template.id!s}/generate-document/",
data,
format="json",
)
assert response.status_code == 401
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
def test_api_templates_generate_document_authenticated_public():
"""Authenticated users can generate pdf document with public templates."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
template = factories.TemplateFactory(is_public=True)
data = {"body": "# Test markdown body"}
response = client.post(
f"/api/v1.0/templates/{template.id!s}/generate-document/",
data,
format="json",
)
assert response.status_code == 200
assert response.headers["content-type"] == "application/pdf"
def test_api_templates_generate_document_authenticated_not_public():
"""
Authenticated users should not be allowed to generate pdf document with templates
that are not marked as public.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
template = factories.TemplateFactory(is_public=False)
data = {"body": "# Test markdown body"}
response = client.post(
f"/api/v1.0/templates/{template.id!s}/generate-document/",
data,
format="json",
)
assert response.status_code == 403
assert response.json() == {
"detail": "You do not have permission to perform this action."
}
@pytest.mark.parametrize("via", VIA)
def test_api_templates_generate_document_related(via, mock_user_teams):
"""Users related to a template can generate pdf document."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
access = None
if via == USER:
access = factories.UserTemplateAccessFactory(user=user)
elif via == TEAM:
mock_user_teams.return_value = ["lasuite", "unknown"]
access = factories.TeamTemplateAccessFactory(team="lasuite")
data = {"body": "# Test markdown body"}
response = client.post(
f"/api/v1.0/templates/{access.template_id!s}/generate-document/",
data,
format="json",
)
assert response.status_code == 200
assert response.headers["content-type"] == "application/pdf"
def test_api_templates_generate_document_type_html():
"""Generate pdf document with the body type html."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
template = factories.TemplateFactory(is_public=True)
data = {"body": "<p>Test body</p>", "body_type": "html"}
response = client.post(
f"/api/v1.0/templates/{template.id!s}/generate-document/",
data,
format="json",
)
assert response.status_code == 200
assert response.headers["content-type"] == "application/pdf"
def test_api_templates_generate_document_type_markdown():
"""Generate pdf document with the body type markdown."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
template = factories.TemplateFactory(is_public=True)
data = {"body": "# Test markdown body", "body_type": "markdown"}
response = client.post(
f"/api/v1.0/templates/{template.id!s}/generate-document/",
data,
format="json",
)
assert response.status_code == 200
assert response.headers["content-type"] == "application/pdf"
def test_api_templates_generate_document_type_unknown():
"""Generate pdf document with the body type unknown."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
template = factories.TemplateFactory(is_public=True)
data = {"body": "# Test markdown body", "body_type": "unknown"}
response = client.post(
f"/api/v1.0/templates/{template.id!s}/generate-document/",
data,
format="json",
)
assert response.status_code == 400
assert response.json() == {
"body_type": [
'"unknown" is not a valid choice.',
]
}
def test_api_templates_generate_document_export_docx():
"""Generate pdf document with the body type html."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
template = factories.TemplateFactory(is_public=True)
data = {"body": "<p>Test body</p>", "body_type": "html", "format": "docx"}
response = client.post(
f"/api/v1.0/templates/{template.id!s}/generate-document/",
data,
format="json",
)
assert response.status_code == 200
assert (
response.headers["content-type"]
== "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
)

View File

@@ -187,9 +187,9 @@ def test_api_templates_list_order_default():
response_template_ids = [template["id"] for template in response_data["results"]]
template_ids.reverse()
assert (
response_template_ids == template_ids
), "created_at values are not sorted from newest to oldest"
assert response_template_ids == template_ids, (
"created_at values are not sorted from newest to oldest"
)
def test_api_templates_list_order_param():
@@ -215,6 +215,6 @@ def test_api_templates_list_order_param():
response_template_ids = [template["id"] for template in response_data["results"]]
assert (
response_template_ids == templates_ids
), "created_at values are not sorted from oldest to newest"
assert response_template_ids == templates_ids, (
"created_at values are not sorted from oldest to newest"
)

View File

@@ -20,6 +20,7 @@ pytestmark = pytest.mark.django_db
CRISP_WEBSITE_ID="123",
FRONTEND_THEME="test-theme",
MEDIA_BASE_URL="http://testserver/",
POSTHOG_KEY={"id": "132456", "host": "https://eu.i.posthog-test.com"},
SENTRY_DSN="https://sentry.test/123",
)
@pytest.mark.parametrize("is_authenticated", [False, True])
@@ -38,8 +39,14 @@ def test_api_config(is_authenticated):
"CRISP_WEBSITE_ID": "123",
"ENVIRONMENT": "test",
"FRONTEND_THEME": "test-theme",
"LANGUAGES": [["en-us", "English"], ["fr-fr", "French"], ["de-de", "German"]],
"LANGUAGES": [
["en-us", "English"],
["fr-fr", "Français"],
["de-de", "Deutsch"],
["nl-nl", "Nederlands"],
],
"LANGUAGE_CODE": "en-us",
"MEDIA_BASE_URL": "http://testserver/",
"POSTHOG_KEY": {"id": "132456", "host": "https://eu.i.posthog-test.com"},
"SENTRY_DSN": "https://sentry.test/123",
}

View File

@@ -24,7 +24,7 @@ def test_api_users_list_anonymous():
def test_api_users_list_authenticated():
"""
Authenticated users should be able to list users.
Authenticated users should not be able to list users without a query.
"""
user = factories.UserFactory()
@@ -37,13 +37,14 @@ def test_api_users_list_authenticated():
)
assert response.status_code == 200
content = response.json()
assert len(content["results"]) == 3
assert content == []
def test_api_users_list_query_email():
"""
Authenticated users should be able to list users
and filter by email.
Authenticated users should be able to list users and filter by email.
Only results with a Levenstein distance less than 3 with the query should be returned.
We want to match by Levenstein distance because we want to prevent typing errors.
"""
user = factories.UserFactory()
@@ -51,70 +52,114 @@ def test_api_users_list_query_email():
client.force_login(user)
dave = factories.UserFactory(email="david.bowman@work.com")
nicole = factories.UserFactory(email="nicole_foole@work.com")
frank = factories.UserFactory(email="frank_poole@work.com")
factories.UserFactory(email="heywood_floyd@work.com")
factories.UserFactory(email="nicole.bowman@work.com")
response = client.get(
"/api/v1.0/users/?q=david.bowman@work.com",
)
assert response.status_code == 200
user_ids = [user["id"] for user in response.json()["results"]]
user_ids = [user["id"] for user in response.json()]
assert user_ids == [str(dave.id)]
response = client.get("/api/v1.0/users/?q=oole")
response = client.get(
"/api/v1.0/users/?q=davig.bovman@worm.com",
)
assert response.status_code == 200
user_ids = [user["id"] for user in response.json()["results"]]
assert user_ids == [str(nicole.id), str(frank.id)]
user_ids = [user["id"] for user in response.json()]
assert user_ids == [str(dave.id)]
response = client.get(
"/api/v1.0/users/?q=davig.bovman@worm.cop",
)
assert response.status_code == 200
user_ids = [user["id"] for user in response.json()]
assert user_ids == []
def test_api_users_list_query_email_matching():
"""While filtering by email, results should be filtered and sorted by similarity"""
def test_api_users_list_limit(settings):
"""
Authenticated users should be able to list users and the number of results
should be limited to 10.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
alice = factories.UserFactory(email="alice.johnson@example.gouv.fr")
factories.UserFactory(email="jane.smith@example.gouv.fr")
michael_wilson = factories.UserFactory(email="michael.wilson@example.gouv.fr")
factories.UserFactory(email="david.jones@example.gouv.fr")
michael_brown = factories.UserFactory(email="michael.brown@example.gouv.fr")
factories.UserFactory(email="sophia.taylor@example.gouv.fr")
# Use a base name with a length equal 5 to test that the limit is applied
base_name = "alice"
for i in range(15):
factories.UserFactory(email=f"{base_name}.{i}@example.com")
response = client.get(
"/api/v1.0/users/?q=michael.johnson@example.gouv.f",
"/api/v1.0/users/?q=alice",
)
assert response.status_code == 200
user_ids = [user["id"] for user in response.json()["results"]]
assert user_ids == [str(michael_wilson.id)]
response = client.get("/api/v1.0/users/?q=michael.johnson@example.gouv.fr")
assert len(response.json()) == 5
# if the limit is changed, all users should be returned
settings.API_USERS_LIST_LIMIT = 100
response = client.get(
"/api/v1.0/users/?q=alice",
)
assert response.status_code == 200
user_ids = [user["id"] for user in response.json()["results"]]
assert user_ids == [str(michael_wilson.id), str(alice.id), str(michael_brown.id)]
assert len(response.json()) == 15
def test_api_users_list_throttling_authenticated(settings):
"""
Authenticated users should be throttled.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
settings.REST_FRAMEWORK["DEFAULT_THROTTLE_RATES"]["user_list_burst"] = "3/minute"
for _i in range(3):
response = client.get(
"/api/v1.0/users/?q=alice",
)
assert response.status_code == 200
response = client.get(
"/api/v1.0/users/?q=ajohnson@example.gouv.f",
"/api/v1.0/users/?q=alice",
)
assert response.status_code == 200
user_ids = [user["id"] for user in response.json()["results"]]
assert user_ids == [str(alice.id)]
assert response.status_code == 429
def test_api_users_list_query_email_matching():
"""While filtering by email, results should be filtered and sorted by Levenstein distance."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
user1 = factories.UserFactory(email="alice.johnson@example.gouv.fr")
user2 = factories.UserFactory(email="alice.johnnson@example.gouv.fr")
user3 = factories.UserFactory(email="alice.kohlson@example.gouv.fr")
user4 = factories.UserFactory(email="alicia.johnnson@example.gouv.fr")
user5 = factories.UserFactory(email="alicia.johnnson@example.gov.uk")
factories.UserFactory(email="alice.thomson@example.gouv.fr")
response = client.get(
"/api/v1.0/users/?q=michael.wilson@example.gouv.f",
"/api/v1.0/users/?q=alice.johnson@example.gouv.fr",
)
assert response.status_code == 200
user_ids = [user["id"] for user in response.json()["results"]]
assert user_ids == [str(michael_wilson.id)]
user_ids = [user["id"] for user in response.json()]
assert user_ids == [str(user1.id), str(user2.id), str(user3.id), str(user4.id)]
response = client.get("/api/v1.0/users/?q=alicia.johnnson@example.gouv.fr")
assert response.status_code == 200
user_ids = [user["id"] for user in response.json()]
assert user_ids == [str(user4.id), str(user2.id), str(user1.id), str(user5.id)]
def test_api_users_list_query_email_exclude_doc_user():
"""
Authenticated users should be able to list users
and filter by email and exclude users who have access to a document.
Authenticated users should be able to list users while filtering by email
and excluding users who have access to a document.
"""
user = factories.UserFactory()
document = factories.DocumentFactory()
@@ -122,17 +167,59 @@ def test_api_users_list_query_email_exclude_doc_user():
client = APIClient()
client.force_login(user)
nicole = factories.UserFactory(email="nicole_foole@work.com")
frank = factories.UserFactory(email="frank_poole@work.com")
nicole_fool = factories.UserFactory(email="nicole_fool@work.com")
nicole_pool = factories.UserFactory(email="nicole_pool@work.com")
factories.UserFactory(email="heywood_floyd@work.com")
factories.UserDocumentAccessFactory(document=document, user=frank)
factories.UserDocumentAccessFactory(document=document, user=nicole_pool)
response = client.get("/api/v1.0/users/?q=oole&document_id=" + str(document.id))
response = client.get(
"/api/v1.0/users/?q=nicole_fool@work.com&document_id=" + str(document.id)
)
assert response.status_code == 200
user_ids = [user["id"] for user in response.json()["results"]]
assert user_ids == [str(nicole.id)]
user_ids = [user["id"] for user in response.json()]
assert user_ids == [str(nicole_fool.id)]
def test_api_users_list_query_short_queries():
"""
Queries shorter than 5 characters should return an empty result set.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.UserFactory(email="john.doe@example.com")
factories.UserFactory(email="john.lennon@example.com")
response = client.get("/api/v1.0/users/?q=jo")
assert response.status_code == 200
assert response.json() == []
response = client.get("/api/v1.0/users/?q=john")
assert response.status_code == 200
assert response.json() == []
response = client.get("/api/v1.0/users/?q=john.")
assert response.status_code == 200
assert len(response.json()) == 2
def test_api_users_list_query_inactive():
"""Inactive users should not be listed."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.UserFactory(email="john.doe@example.com", is_active=False)
lennon = factories.UserFactory(email="john.lennon@example.com")
response = client.get("/api/v1.0/users/?q=john.")
assert response.status_code == 200
user_ids = [user["id"] for user in response.json()]
assert user_ids == [str(lennon.id)]
def test_api_users_retrieve_me_anonymous():
@@ -163,6 +250,7 @@ def test_api_users_retrieve_me_authenticated():
"id": str(user.id),
"email": user.email,
"full_name": user.full_name,
"language": user.language,
"short_name": user.short_name,
}

View File

@@ -0,0 +1,94 @@
"""
Unit tests for the filter_root_paths utility function.
"""
from core.api.utils import filter_root_paths
def test_api_utils_filter_root_paths_success():
"""
The `filter_root_paths` function should correctly identify root paths
from a given list of paths.
This test uses a list of paths with missing intermediate paths to ensure that
only the minimal set of root paths is returned.
"""
paths = [
"0001",
"00010001",
"000100010001",
"000100010002",
# missing 00010002
"000100020001",
"000100020002",
"0002",
"00020001",
"00020002",
# missing 0003
"00030001",
"000300010001",
"00030002",
# missing 0004
# missing 00040001
# missing 000400010001
# missing 000400010002
"000400010003",
"0004000100030001",
"000400010004",
]
filtered_paths = filter_root_paths(paths, skip_sorting=True)
assert filtered_paths == [
"0001",
"0002",
"00030001",
"00030002",
"000400010003",
"000400010004",
]
def test_api_utils_filter_root_paths_sorting():
"""
The `filter_root_paths` function should fail is sorting is skipped and paths are not sorted.
This test verifies that when sorting is skipped, the function respects the input order, and
when sorting is enabled, the result is correctly ordered and minimal.
"""
paths = [
"0001",
"00010001",
"000100010001",
"000100020002",
"000100010002",
"000100020001",
"00020001",
"0002",
"00020002",
"000300010001",
"00030001",
"00030002",
"0004000100030001",
"000400010003",
"000400010004",
]
filtered_paths = filter_root_paths(paths, skip_sorting=True)
assert filtered_paths == [
"0001",
"00020001",
"0002",
"000300010001",
"00030001",
"00030002",
"0004000100030001",
"000400010003",
"000400010004",
]
filtered_paths = filter_root_paths(paths)
assert filtered_paths == [
"0001",
"0002",
"00030001",
"00030002",
"000400010003",
"000400010004",
]

View File

@@ -0,0 +1,107 @@
"""Unit tests for the nest_tree utility function."""
import pytest
from core.api.utils import nest_tree
def test_api_utils_nest_tree_empty_list():
"""Test that an empty list returns an empty nested structure."""
# pylint: disable=use-implicit-booleaness-not-comparison
assert nest_tree([], 4) is None
def test_api_utils_nest_tree_single_document():
"""Test that a single document is returned as the only root element."""
documents = [{"id": "1", "path": "0001"}]
expected = {"id": "1", "path": "0001", "children": []}
assert nest_tree(documents, 4) == expected
def test_api_utils_nest_tree_multiple_root_documents():
"""Test that multiple root-level documents are correctly added to the root."""
documents = [
{"id": "1", "path": "0001"},
{"id": "2", "path": "0002"},
]
with pytest.raises(
ValueError,
match="More than one root element detected.",
):
nest_tree(documents, 4)
def test_api_utils_nest_tree_nested_structure():
"""Test that documents are correctly nested based on path levels."""
documents = [
{"id": "1", "path": "0001"},
{"id": "2", "path": "00010001"},
{"id": "3", "path": "000100010001"},
{"id": "4", "path": "00010002"},
]
expected = {
"id": "1",
"path": "0001",
"children": [
{
"id": "2",
"path": "00010001",
"children": [{"id": "3", "path": "000100010001", "children": []}],
},
{"id": "4", "path": "00010002", "children": []},
],
}
assert nest_tree(documents, 4) == expected
def test_api_utils_nest_tree_siblings_at_same_path():
"""
Test that sibling documents with the same path are correctly grouped under the same parent.
"""
documents = [
{"id": "1", "path": "0001"},
{"id": "2", "path": "00010001"},
{"id": "3", "path": "00010002"},
]
expected = {
"id": "1",
"path": "0001",
"children": [
{"id": "2", "path": "00010001", "children": []},
{"id": "3", "path": "00010002", "children": []},
],
}
assert nest_tree(documents, 4) == expected
def test_api_utils_nest_tree_decreasing_path_resets_parent():
"""Test that a document at a lower path resets the parent assignment correctly."""
documents = [
{"id": "1", "path": "0001"},
{"id": "6", "path": "00010001"},
{"id": "2", "path": "00010002"}, # unordered
{"id": "5", "path": "000100010001"},
{"id": "3", "path": "000100010002"},
{"id": "4", "path": "00010003"},
]
expected = {
"id": "1",
"path": "0001",
"children": [
{
"id": "6",
"path": "00010001",
"children": [
{"id": "5", "path": "000100010001", "children": []},
{"id": "3", "path": "000100010002", "children": []},
],
},
{
"id": "2",
"path": "00010002",
"children": [],
},
{"id": "4", "path": "00010003", "children": []},
],
}
assert nest_tree(documents, 4) == expected

Some files were not shown because too many files have changed in this diff Show More