Compare commits

..

353 Commits

Author SHA1 Message Date
Anthony LC
bb9895d63f tag-v0.1.0-coming-soon 2024-06-21 12:45:11 +02:00
Anthony LC
6a053b19fb (frontend) coming soon
Add a coming soon page bind to the
regie.numerique.gouv.fr domain.
2024-06-21 12:45:11 +02:00
Anthony LC
c6b1eb12cc 🎨(frontend) create PageLayout
- Create PageLayout, a simple layout for pages
that don't need a sidebar and don't need to be
contains in the screen height.
- Moves the layouts in the layouts folder.
2024-06-21 12:45:11 +02:00
Anthony LC
f8bce32b93 🚚(helm) move secret to desk/templates
With the recent changes to the helm chart,
the secrets.yaml file was not found by
Tilt anymore. This commit moves the file
to the correct location.
2024-06-21 12:43:24 +02:00
Anthony LC
a373704569 👷(CI) add production tag to deploy workflow
Add `production` tag to deploy workflow.
Every tag production will trigger
the deploy workflow to production environment.
2024-06-21 12:38:05 +02:00
Anthony LC
4cbe3c8a49 👷(helm) prod configuration
Add the prod configuration to the helm chart.
2024-06-21 12:38:05 +02:00
Jacques ROUSSEL
6d0dbf0d13 💚(CI) upgrade submodule
- fix secrets for staging
2024-06-20 14:17:22 +02:00
Jacques ROUSSEL
4048855b1b 💚(CI) upgrade submodule
- upgrade submodule to use new preprod secret
2024-06-19 17:26:52 +02:00
Anthony LC
e2a682cae4 ⬇️(frontend) downgrading fetch-mock to 9.11.0
fetch-mock relesed the version 10, but it is
causing some issues in the frontend tests for
the moment. The adoption is still low, the
documentation is not updated, so let's
downgrade it to 9.11.0 for now.
2024-06-19 15:41:20 +02:00
Anthony LC
366f10689b (frontend) add "@testing-library/dom
Recent update of @testing-library/react requires
@testing-library/dom to be installed as well.
2024-06-19 15:41:20 +02:00
renovate[bot]
922719b13e ⬆️(dependencies) update js dependencies 2024-06-19 15:41:20 +02:00
Anthony LC
3c481e75bb 👷(helm) command createsuperuser
We need a superuser in the Django
application, to have access to the admin part.
This commit create a superuser on the pods.
2024-06-19 13:34:15 +02:00
Anthony LC
9a7a8e4a34 🔥(helm) remove uneeded file
secrets.yaml was duplicated in the helm chart,
we can remove this one.
2024-06-18 15:40:33 +02:00
Anthony LC
905b673413 💚(CI) upgrade submodule
- Change submodule ref to get preprod secret
2024-06-18 15:40:33 +02:00
Anthony LC
21981c6478 💚(CI) remove trigger workflow on push tags
We were starting the workflow on push tags,
it is needed for the docker-hub workflow,
but the other workflows does not need to
be triggered on push tags.
2024-06-18 15:40:33 +02:00
Anthony LC
e56c63676e 👷(CI) add deploy workflow
Add the deploy workflow, this workflow will deploy
the application to the selected tag.
2024-06-18 15:40:33 +02:00
Anthony LC
187005d441 👷(helm) preprod configuration
Add the preprod configuration to the helm chart.
2024-06-18 15:40:33 +02:00
Anthony LC
54b7a637fe 🔧(backend) activate https on oidc redirection
mozilla-django-oidc didn't add the `https`
prefix to the redirect_uri.
We set the option SECURE_PROXY_SSL_HEADER to
('HTTP_X_FORWARDED_PROTO', 'https') in the
settings.py file to force the https prefix.
2024-06-18 15:40:33 +02:00
renovate[bot]
35a897fa60 ⬆️(dependencies) update python dependencies 2024-06-16 23:55:07 +02:00
Sabrina Demagny
b4bafb6efb (mailbox_manager) modify API to get maildomain
Access to maildomain by slug name
2024-06-13 15:10:04 +02:00
Jacques ROUSSEL
23778fda0d 💚(ci) improve submodule usage
- remove deplucate declaration
- simplify helmfile
- use symlink
2024-06-11 10:46:40 +02:00
daproclaima
0a8c488649 🧑‍💻(frontend) run automatically prettier on translations.json
The frontend translations.json could be validated by
the eslint scripts without running manually
prettier. This commit fixes this by running prettier
script automatically once the frontend translations
are extracted.
2024-06-10 12:38:19 +02:00
Anthony LC
e6b5f32a61 🐛(i18n) comma in keys
Having a comma in the keys broke the parser.
This commit allows the comma in keys.
2024-06-10 12:38:19 +02:00
daproclaima
8af47283c8 💬(app-desk) update translations of teams feature
This commit is part of the replacement of "teams" by "groups" wording task.
It changes words in the "teams" feature and add more meaningful wording in
English and French, and removes the icon used in the title of the group
member deletion modal as it rather lets think we are deleting a group.
Updates of related e2e and rendering tests come along with these changes.
2024-06-10 12:38:19 +02:00
Jacques ROUSSEL
8a44718e6b 💚(ci) fix
- fix broken front push docker image
2024-06-07 17:09:55 +02:00
Jacques ROUSSEL
6e7f20eda9 💚(ci) remove secret from repository
- Remove *.enc.*
- Adapt helmfile
- Adapt CI
2024-06-07 16:30:14 +02:00
Sabrina Demagny
b3779b5979 🧑‍💻(makefile) improve django migration commands
Add showmigrations command.
Allow to pass args to migrate and makemigrations commands.
2024-06-06 10:30:26 +02:00
Anthony LC
4b80b288f9 ♻️(mails) link email from current site
The link in the email was pointing on the
staging website. We now use a variable to
target the current site setup in the database.
2024-06-05 09:50:09 +02:00
Anthony LC
fbec4af261 🧑‍💻(mail) make commands windows friendly
Make the commands windows friendly.
2024-06-05 09:50:09 +02:00
Anthony LC
b8499a539e 🐛(frontend) fix duplicate call on getMe
Fix duplicate call on getMe.
The logout process was refactored recently,
the hook dependency is not necessary anymore and
was creating a duplicate call on getMe.
2024-06-04 11:52:36 +02:00
Anthony LC
c7d1312f89 ♻️(frontend) frontend environment free
Until now, the front had to know at build time
the url of the backend and the webrtc server
to be able to communicate with them.
It is not optimal because it means that we need
multiple docker image (1 per environment) to have
the app working, it is not very flexible.

This commit will make the frontend "environment free"
by determining these urls at runtime.
2024-06-04 11:52:36 +02:00
Anthony LC
4636c611c6 🔧(helm) add namespace to the templates
The goal of adding a namespace in the templates
is to ensure that resources are deployed
in a specific, possibly isolated part of the Kubernetes cluster.
This helps in organizing resources, managing
permissions, and applying configurations or
limits appropriately within the cluster.
2024-06-04 10:52:17 +02:00
Anthony LC
211d89cae0 🔨(CI) add Tilt
Tilt is a tool for local Kubernetes development.
It makes it easy to see your changes as you
make them, and it rebuilds and redeploys
your app as you change it.
2024-06-04 10:52:17 +02:00
Anthony LC
915731e218 💚(ci) improve secrets for k8s deployment
Avoid secrets to be visible from running deployments
2024-06-04 10:52:17 +02:00
lebaudantoine
c534048e97 🔧(compose) stop forcing platform for Keycloak PostgreSQL image
Forcing `platform: linux/amd64` for the PostgreSQL image causes compatibility
issues and performance degradation on Mac ARM chips (M1/M2). Removing the
platform specification allows Docker to select the appropriate architecture
automatically, ensuring better performance and compatibility.
2024-06-03 13:56:28 +02:00
renovate[bot]
5d1e2bd39d ⬆️(dependencies) update python dependencies 2024-06-03 09:49:51 +02:00
Jacques ROUSSEL
67d3e58c82 🐛(ci) improve docker-hub
Avoid to notify argocd for nothing
2024-05-31 17:08:59 +02:00
antoine lebaud
e0739689e6 🚨(backend) handle new checks introduced in Pylint v3.2.0
Pylint 3.2.0 introduced a new check `possibly-used-before-assignment`, which
ensures variables are defined regardless of conditional statements.

Some if/else branches were missing defaults. These have been fixed.
2024-05-31 12:53:11 +02:00
renovate[bot]
04717fd629 ⬆️(dependencies) update python dependencies 2024-05-31 12:53:11 +02:00
Lebaud Antoine
087bbf74f6 🔧(helm) setup logout flow from Agent Connect
Add the relevant environment configurations to make sure the backend
in dev and staging environments log out the user from Agent Connect.
2024-05-31 12:14:58 +02:00
Lebaud Antoine
63a875bd5b ♻️(frontend) redirect the user agent to the logout endpoint
Recent updates in the backend views now requires the user agent to be
redirected to the logout endpoint.

The logout endpoint should initiate the logout flow with the OIDC provider,
by redirecting the user to the OIDC provider domain.

Thus, OIDC provider session cookie should be cleared.

E2E tests should be improved later on, when the CI and the development env
use Agent Connect integration environment. The current logout is not working
with the Keycloack configuration.
2024-05-31 12:14:58 +02:00
Lebaud Antoine
7a26f377e3 (backend) support Agent Connect Logout flow
The default Logout view provided by Mozilla Django OIDC is not suitable
for the Agent Connect Logout flow.

Previously, when a user was logging-out, only its Django session was ended.
However, its session in the OIDC provider was still active.

Agent Connect implements a 'session/end' endpoint, that allows services to
end user session when they logout.

Agent Connect logout triggers cannot work with the default views implemented
by the dependency Mozilla Django OIDC. In their implementation, they decided
to end Django Session before redirecting to the OIDC provider.

The Django session needs to be retained during the logout process.

An OIDC state is saved to the request session, pass to Agent Connect Logout
endpoint, and verified when the backend receives the Logout callback from Agent
Connect. It seems to follow OIDC specifications.

If for any reason, the Logout flow cannot be initiated with Agent Connect,
(missing ID token in cache, unauthenticated user, etc), the user is redirected
to the final URL, without interacting with Agent Connect.
2024-05-31 12:14:58 +02:00
Lebaud Antoine
05d9a09d63 🚚(backend) create a dedicated authentication package
Prepare adding advanced authentication features. Create a dedicated
authentication Python package within the core app.

This code organization will be more extensible.
2024-05-31 12:14:58 +02:00
daproclaima
735db606f6 🚚(app-desk) rename useMailDomainMailboxes into useMailboxes
Shortens the hook name and update its imports and exports.
2024-05-29 16:11:59 +02:00
daproclaima
2bf85539f1 (e2e) add mailbox creation tests
Checks user can create a mailbox for a mail domain
and that form fields are visually interactive
according to the form validation state.
2024-05-29 16:11:59 +02:00
daproclaima
6981ef17df (app-desk) create mailbox for a mail domain
Add form to create a mailbox for a mail domain. It sends a http
POST request mail-domains/<mail-domain-id>/mailboxes/ on form
submit. The form appears inside a modal.
Installs react-hook-form, zod, and @hookform/resolvers for form
manipulation and field validation.
2024-05-29 16:11:59 +02:00
daproclaima
37d32888f5 (app-desk) displays specific mail domain mailboxes
Fetches a mail domain by id and displays
its mailboxes as a list in a table.
Associated with e2e tests.
2024-05-24 12:10:44 +02:00
daproclaima
d4e0f74d30 (app-desk) add /mail-domains/<id> page
Adds a page in charge of finding the mail domain
matching the id provided in url query params.
In the future the domain name will be used.
2024-05-24 12:10:44 +02:00
Anthony LC
d2c7eaaa4b 🐛(app-desk) fix fetchPriority warning
The upgrade to react@18.3.1 has a compatibility
issue with next@14.2.3. It creates a error warning
about the fetchPriority prop. This commit fixes the
issue by downgrading react to 18.2.0 as it was
before the last upgrade.
2024-05-13 17:10:37 +02:00
Anthony LC
46aaf7351d ♻️(app-desk) adapt La Gaufre
Adapt La Gaufre with the new configuration.
2024-05-13 17:10:37 +02:00
daproclaima
76e9d58b6c (app-desk) add sorting to mail domains panel
Adds a button to sort items in the mail domains panel
by creation date. Also adds an e2e test.
2024-05-13 16:23:14 +02:00
daproclaima
9b198d0bab (app-desk) add panel for mail domains
Adds a panel based on teams' one. It fetches all mail-domains
the connected user has relationships with and displays them
as a list of links redirecting to
/mail-domains/<my-domain-name>. Updates e2e tests accordingly.
2024-05-13 16:23:14 +02:00
daproclaima
3f1b446e8e 🚚(app-desk) application mail renamed into mail-domains
Rename any folder or file containing mails into mail-domains.
Update all url redirection links accordingly.
2024-05-13 16:23:14 +02:00
Anthony LC
5049c9b732 ️(frontend) clean yarn.lock
Some compatibility issues with dependencies were
fixed by cleaning the yarn.lock.
2024-05-13 14:54:31 +02:00
renovate[bot]
7f2adb8d2f ⬆️(dependencies) update js dependencies 2024-05-13 14:54:31 +02:00
renovate[bot]
b12992f125 ⬆️(dependencies) update python dependencies 2024-05-09 23:15:12 +02:00
Anthony LC
001673f973 💬(app-desk) change some texts
Change some texts on the team page.
2024-05-06 17:00:39 +02:00
Anthony LC
4b4bbc4c0a 🐛(app-desk) fix fetchPriority warning
The upgrade to react@18.3.1 has a compatibility
issue with next@14.2.3. It creates a error warning
about the fetchPriority prop. This commit fixes the
issue by downgrading react to 18.2.0 as it was
before the last upgrade.
2024-05-06 16:50:17 +02:00
Anthony LC
b7b90d1bf3 💄(app-desk) keep highlighting menu when sub menu
When sub menu was open, the parent menu was not
highlighted.
This commit fixes this issue.
2024-05-06 12:09:17 +02:00
Anthony LC
c599757d7a 🗑️(app-desk) clean the menu
- remove unused icons
- remove unused pages
- remove menu items
2024-05-06 12:09:17 +02:00
renovate[bot]
a0992b6ba9 ⬆️(dependencies) update js dependencies 2024-05-06 10:27:25 +02:00
Anthony LC
d88f6a5a51 ♻️(app-desk) replace classname spacings
Replace the classname spacings with the new
spacing system based on props.
2024-04-30 11:42:26 +02:00
Anthony LC
a45408c93c 🎨(app-desk) add margin and padding to Box
Add margin and padding system to Box component.
It proposes the autocompletion.
It is bind with the Cunninghams spacing system.
2024-04-30 11:42:26 +02:00
Anthony LC
c94888fe09 ⬇️(frontend) react 18.3.1 -> 18.2.0
Downgrade react to 18.2.0.
It seems to have a compatibility issue
with @openfun/cunningham-react.
2024-04-29 10:27:15 +02:00
Anthony LC
4551d20d67 🔥(eslint) remove uneeded yarn.lock file
The yarn.lock doesn't seem necessary for this
package, so we're removing it.
2024-04-29 10:27:15 +02:00
Anthony LC
c7f257daa0 ⬇️(eslint-config-people) eslint 9.0.0 -> 8.57.0
Downgrade eslint to 8.57.0.
9.0.0 has breaking changes, the adoption
is still very low, better to wait.
Add it in the renovate.json file.
2024-04-29 10:27:15 +02:00
renovate[bot]
32e6996b68 ⬆️(dependencies) update js dependencies 2024-04-29 10:27:15 +02:00
Jacques ROUSSEL
8fbc4e936e 💚(ci) improve secrets for k8s deployment
Avoid secrets to be visible from running deployments
2024-04-23 22:19:25 +02:00
renovate[bot]
cda59fecec ⬆️(dependencies) update python dependencies 2024-04-22 13:46:27 +02:00
Marie PUPO JEAMMET
c5ba87e422 📝(doc) copy demo credentials to README.md
Copy demo admin credentials for newcomers to log in to django admin
without having to find it in makefile.
2024-04-19 18:45:50 +02:00
Marie PUPO JEAMMET
df24c24da1 (api) add CRUD for mailbox manager MailDomain models
Add create,list,retrieve and delete actions for MailDomain model.
2024-04-19 18:45:50 +02:00
Marie PUPO JEAMMET
ac81e86c88 🧑‍💻(admin) add mailbox-related models to django admin
Register MailDomain, MailDomainAccess and Mailbox to django admin.
2024-04-18 10:42:13 +02:00
Sabrina Demagny
082fb99bd5 (api) allow to list and create Mailboxes
Simply display all Mailboxes create for a MailDomain.
LDAP connection is not yet available, it will be implemented soon.
Read and create permissions will be refined soon too.
2024-04-17 16:51:54 +02:00
renovate[bot]
1704ba1707 ⬆️(dependencies) update gunicorn to v22 [SECURITY] 2024-04-17 11:23:11 +02:00
Marie PUPO JEAMMET
cca6c77f00 🗃️(models) add MailDomain, MailDomainAccess and Mailbox models
Additional app and models to handle email addresses creation in Desk.
2024-04-16 15:47:33 +02:00
renovate[bot]
a1f9cf0854 ⬆️(dependencies) update python dependencies 2024-04-16 10:27:16 +02:00
Lebaud Antoine
2f1805b721 🩹(backend) address linter flakiness on Email tests
Pylint was randomly failing due to a warning while unpacking emails.
The W0632 (Possible unbalanced tuple unpacking) was triggered.

Replace tuple unpacking by an explicitly accessing the first element of
the array using index.
2024-04-08 15:35:12 +02:00
renovate[bot]
711abcb49f ⬆️(dependencies) update python dependencies 2024-04-08 15:35:12 +02:00
Anthony LC
e68370bfcd ⬇️(eslint-config-people) eslint 9.0.0 -> 8.57.0
Downgrade eslint to 8.57.0.
9.0.0 has breaking changes, the adoption
is still very low (1%), better to wait.
2024-04-08 15:18:17 +02:00
renovate[bot]
ae1acd8840 ⬆️(dependencies) update js dependencies 2024-04-08 15:18:17 +02:00
Lebaud Antoine
54386fcdd3 🩹(backend) address test flakiness while sorting Team accesses
Previously, there was a difference between Django's `order_by`
behavior and Python's `sorted` function, leading to test failures
under specific conditions. For example, entries such as 'Jose Smith'
and 'Joseph Walker' were not consistently sorted in the same order
between the two methods.

To resolve this issue, we've ensured that sorting the expected
results in the TeamAccess tests are both case-insensitive and
space-insensitive. This adjustment fix tests flakiness
2024-04-08 15:07:58 +02:00
Anthony LC
45a3e7936d (app-desk) create mails feature
Create the archi to handle the mails feature.
It has a different layout than the other features,
we don't display the sidebar to keep the
user focused on the mail content.
2024-04-08 14:42:56 +02:00
Marie PUPO JEAMMET
ebf58f42c9 (webhook) add webhook logic and synchronization utils
adding webhooks logic to send serialized team memberships data
to a designated serie of webhooks.
2024-04-05 16:06:09 +02:00
Samuel Paccoud - DINUM
7ea6342a01 ♻️(models) refactor user email fields
The email field on the user is renamed to "admin_email" for clarity. The
"email" and "name" fields of user's main identity are made available on
the user model so it is easier to access it.
2024-04-05 16:06:09 +02:00
Anthony LC
6d807113bc 🔧(sops) update secrets
Access to anthony's new key
2024-04-05 12:21:13 +02:00
Jacques ROUSSEL
5455c589ef 🔧(sops) update secrets
Decrypt and reencrypt secrets to grant access to anthony's new key
2024-04-05 09:48:19 +02:00
Lebaud Antoine
9ec7eddaed (frontend) add logout button
Rework the header based on latest Johann's design, which introduced a
dropdown menu to mange user account.

In this menu, you can find a logout button, which ends up the backend
session by calling the logout endpoint. Please that automatic redirection
when receiving the backend response were disabled. We handle it in our
custom hook, which reload the page.

Has the session cookie have been cleared, on reloading the page, a new
loggin flow is initiated, and the user is redirected to the OIDC provider.

Please note, the homepage design/organization is still under discussion, I
prefered to ship a first increment. The logout feature will be quite useful
in staging to play and test our UI.
2024-04-04 14:52:53 +02:00
Lebaud Antoine
7db2faa072 🍱(frontend) rename Desk to Equipes
Update all assets related to the previous naming.
I gave some asset a more generic name, so if the app's name chang again
we won't have to rename the logo.

Linked but not assets, meta title and description were updated.
Next.js favicon was replaced by the Equipes' one.
2024-04-04 14:52:53 +02:00
Anthony LC
6e0b329b09 🐛(app-desk) add next-env.d.ts
next-env.d.ts lot of types for the next.js.
next.js boilerplate don't version the
next-env.d.ts file, so it was being ignored by git.
This was causing the CI to fail with the ts linter.
2024-04-04 12:13:00 +02:00
Lebaud Antoine
c7aae51d25 🩹(test) update MemberAction props
TeamId prop was refactored to a Team object.
Update MemberAction component's tests.
2024-04-04 12:13:00 +02:00
Anthony LC
db40efb360 🚨(app-desk) improve linter
The linter was passing near the ts errors in
the tests, we improve the linter by adding
a ts checkup.
2024-04-04 12:13:00 +02:00
Anthony LC
3ddc519d9e (e2e) fix job flakinness
Fix a flaky jobs
- searched username could be hidden in the options
 depends the dummy data generated.
- remove first place assertion when create a new team,
  multiple workers make this assertion flaky
2024-04-04 12:13:00 +02:00
Lebaud Antoine
e20960e3e1 💚(ci) update Github Actions using Node.js 16
Github Actions are transitioning from Node 16 to Node 20. Make sure we use
latest Github Actions versions to clean any deprecation warnings.

The migration is upcoming.
2024-04-04 10:33:20 +02:00
Anthony LC
1223732fa9 🐛(CI) improve caching
When we restored the frontend cache, we were restoring
old code as well, we don't want that, we want to only
restore the node_modules.
This commit fixes that.
We improve the build-front caching as well, to cache
only the desk build app.
2024-04-02 16:12:32 +02:00
Anthony LC
e8180bc49b 💄(app-desk) retouch design grid members
The update of Cunningham seems to have lightly
broken the design of the grid members.
This commit fixes the design of the grid members.
2024-04-02 16:12:32 +02:00
Anthony LC
ffe997a658 ️(frontend) clean yarn.lock
The yarn.lock file get full of garbage and old
dependencies after a while. This commit cleans it.
2024-04-02 16:12:32 +02:00
renovate[bot]
d4c23ce5b9 ⬆️(dependencies) update js dependencies 2024-04-02 16:12:32 +02:00
Lebaud Antoine
5ed05b96a5 🩹(frontend) disable submission button while a request is pending
If any request is taking longer than expected, the user could interact with
the frontend, thinking the previous submission was not taken into account,
and would re-submit a request.

It happened to me while sending an invitation. Replaying request can either
lead to inconsistencies in db for the user, or to errors in requests' response.

I propose to disable all interactive button while submitting a request.
It's good enough for this actual sprint, these types of interactivity issues
could be improved later on.
2024-04-02 15:07:59 +02:00
Anthony LC
df15b41a87 🚸(app-desk) cannot select user or email already selected
- Add the teamid to the useUsers query, to not get
  the users that are already in the team
- Add a check to not select a user or email
  that is already selected
2024-04-02 11:45:27 +02:00
Anthony LC
591045b0ec ️(app-desk) clean build pages
Prob:
Next.js transpiles all the files present in the
`pages` directory. But we don't want to transpile
the providers neither the Layout components.

Solution:
We export these components to a core folder.
2024-04-02 11:34:40 +02:00
Sabrina Demagny
775b32ff45 (backend) enhance search users to add in a team
Exclude from the result all users already members of the current team
2024-04-02 11:12:08 +02:00
renovate[bot]
e9a628f816 ⬆️(dependencies) update python dependencies 2024-04-02 11:11:42 +02:00
Anthony LC
bf1450cfa7 ️(app-desk) remove firefox from e2e tests
- Remove Firefox testing, Firefox browser seems unstable with
Playwright, most of the time the failing tests are the one
with Firefox, Firefox is only 3% of the browser.
- Improve some naming in the test creation to avoid
conflit name.
2024-04-02 10:54:04 +02:00
Anthony LC
480d8277cc ️(CI) persist the frontend between jobs
To improve the speed of the CI, we cache the frontend
install. It will even be reused between pull request
until the yarn.lock has a change.
We cache as well the desk build app, in another cache,
this cache persist only per workflow. It will increase the
speed if we have e2e flaky tests and that we have to relaunch
the e2e job.
2024-04-02 10:54:04 +02:00
Lebaud Antoine
97cec8901c 📱(frontend) enhance team info style to avoid layout shift
On small screens, padding and alignment were causing texts to render wrapped.
It might be a personal choice, but I decided to give more space to the text,
thus avoiding text to wrap and ending up in increasing team info's height.

This issue arises when closing / opening the panel menu.
2024-03-29 15:32:52 +01:00
Lebaud Antoine
e8aba29a68 (frontend) make the team pannel closable
Team panel was quite wide, and took too much space on small screens.
Johann decided to make it closable. Thus, user that needs to navigate quickly
between their team can open it, use it and then close it.

This animation is a first draft, and should be improved later on. Also, some
accessibility issues might ariase, I have ignored them in this first draft.
2024-03-29 15:32:52 +01:00
Lebaud Antoine
ebaa1360e7 🍱(frontend) update icon button 'add team' in teams panel
Johann recently changed the icon use by the 'add team' button. He preferred
having a square one to contrast more with the 'collapse menu' icon that will
be added later on.
2024-03-29 15:32:52 +01:00
Anthony LC
6d8e05b746 🚨(app-desk) remove warning next.js about css from Head
Importing the css from Head was causing a flickering
effect on the button, because of the time the css
load. Next.js was emitting as well a warning about
css being loaded from the Head component.
We moved the css import to the _document.tsx file
as recommended by the Next.js documentation.
2024-03-29 14:30:34 +01:00
Sabrina Demagny
e7049632ab 📝(frontend) add missing info in the README
add missing info about how to run front and accesses
2024-03-29 10:05:19 +01:00
daproclaima
a54bcbcb1e 💄(app-desk) integrate design 404 page
Introduce a first draft of 404 page based on Johann's design. Please
refer to the Figma file for more info. This page is tested through
tests e2e. It closes the issue #112
2024-03-27 17:26:54 +01:00
daproclaima
4af4c4de50 🙈(common) ignore .tools-version
add .tools-version to .gitignore as I use
asdf as a package and version manager.
2024-03-27 17:26:54 +01:00
Anthony LC
1c5499b2ab (app-desk) remove warning request not mocked
A request was not mocked in the test,
so the warning a warning was displayed.
This commit mocks the request to avoid the warning.
2024-03-27 14:31:49 +01:00
Anthony LC
91f755306b (app-desk) improve sorting teams test e2e
This tests was becoming very flaky because we create
teams in parallel with the other tests.
We use another approch, we checks the aria are
changing according to the sort, we check
as well the api request and that the response
is ok.
2024-03-27 14:31:49 +01:00
Anthony LC
224025c3fb ♻️(app-desk) add hook useWhoAmI
The hook useWhoAmI is a custom hook that gives informations
about the current member.
2024-03-27 14:31:49 +01:00
Anthony LC
724bbe550c (app-desk) modal delete member
We can now delete a member from a team.
We take care of usecases like:
- it is the last owner of the team (cannot delete)
- other owner of the team (cannot delete)
- role hierarchy
2024-03-27 14:31:49 +01:00
Anthony LC
016232ad2d (app-desk) add useDeleteTeamAccess react-query hook
Add the hook useDeleteTeamAccess, it will be used to
delete member from a team.
2024-03-27 14:31:49 +01:00
Lebaud Antoine
6de24d973b 🔇(helm) silence some Django system checks
Django logs some security warnings we can ignored when deploying over K8s.
Inspired by fun project, I added the Django setting SILENCED_SYSTEM_CHECKS,
and silenced the two that were logging a lot of warning.
2024-03-27 12:14:36 +01:00
Lebaud Antoine
04c107cfdb 🐛(helm) enable SSL when sending email
Email settings were wrongly configured. It led to unsent email and timeout
response from the backend server.

I forgot to enable the SSL while using the Email service from scalingo.
2024-03-27 12:14:36 +01:00
Lebaud Antoine
cbfc67f010 🔒️(helmfile) generate Django secret key
Generate a proper Django secret key ready for production,
using the provided get_random_secret_key() function.

Store its value in a k8s secret. I generated two values one for
dev and one for staging.

Previous values were triggering security logs.
2024-03-27 12:14:36 +01:00
Anthony LC
0fe0175622 💬(app-desk) translate en plurals keys correctly
We added the english language in Crowdin.
We can now translate the plural keys in
the english language.
2024-03-27 07:15:42 +01:00
Anthony LC
fab1329712 🌐(i18n) don't keep unused keys
The potential unused keys was the ones from other
PR, but we don't use crowdin with every PR, so we don't
need to keep the unused keys in the translation files.
2024-03-27 07:15:42 +01:00
Anthony LC
f27b347d1c 💬(frontend) synch the translations with crowdin
We stopped the use Crowdin for every PR, so we need to
synch the translations here and there to be sure
all the translations are up to date.
2024-03-27 07:15:42 +01:00
Lebaud Antoine
cc35757c9e 🚧(frontend) add applications menu PoC
Based on works from @manuhabitela, introduce a PoC of the future component.
ApplicationsMenu component is still under construction.

This code was committed for the Wednesday 26th demo, to showcase our future
works. This Next.js integration could be improved, and will for sure!
Don't blame me.
2024-03-26 22:49:57 +01:00
Lebaud Antoine
a1065031ee (frontend) sort team's members
We are displaying all team's members in a datagrid. This propose to enable
sorting on columns, to easily find members.

Please note that few issues were faced when activating the sorting on the
Cunningham components. First, custom columns can not be sorted (yet), a PR
has been merged on Cunningham's side. We're waiting for the next release.

Second, when sorting data rows, if any of the column has some null values,
the datagrid sorting state becomes inconsistent. Thx @AntoLC for spotting the
issue. It's work in progress on Cunningham's side to fix the issue.

Finally, Cunningham export only the SortModel type, which is an array, and
doesn't export its items' type. I might have miss something but it feels weird
to redefine its items type.

Columns wiggle on sorting, because they data is set to undefined while fetching
the next batch. it's visually weird, but not a major pain.
Next release of Cunningham will allow us to set the column to a fixed size.
2024-03-26 22:15:18 +01:00
Jacques ROUSSEL
7c488a9807 🚀(helm) transform migrate job to Presync job
Apply db migration before syncing all the pods.
It avoids triggering errors when running the migrate job.
2024-03-26 17:45:53 +01:00
Jacques ROUSSEL
1c4efd523b 👷(argocd) notify argocd when new images are pushed
Add a new job in the CI, which notifies ArgoCD through a webhook that a new
docker image has been pushed to the Docker registry. Thus, ArgoCD can sync
and pull the latest image.

Thus, main will be automatically deployed to staging.
2024-03-26 17:01:15 +01:00
Lebaud Antoine
2345250c4f 🚑️(staging) fix 404 errors
Recent changes on the staging cluster created a regression.
The ingress className needs to be specified.
2024-03-26 16:39:48 +01:00
Anthony LC
6b2fb4169c 🛂(app-desk) add TeamActions component
TeamActions is a component that control the actions
that a member can performed on the team.
It contains a dropdown menu that contains the actions:
- Edit Team
- Remove Team
2024-03-26 16:28:51 +01:00
Anthony LC
73e58e274c (app-desk) modal delete team
We can now delete a team.
2024-03-26 16:28:51 +01:00
Anthony LC
55fbd661b0 (app-desk) add useRemoveTeam react-query hook
Add the hook useRemoveTeam, it will be used to
remove a team.
2024-03-26 16:28:51 +01:00
Anthony LC
f591c95a92 🏷️(app-desk) move Role type to Team
We need the Role type in Team as well.
Better to move it to the highest level.
2024-03-26 16:28:51 +01:00
Anthony LC
25af872a2a 🚚(app-desk) create addMembers feature
All the code related to adding members has been moved
to the addMembers feature.
2024-03-25 18:08:44 +01:00
Anthony LC
832dae789e (app-desk) add new member to team
We integrate the endpoint to add a new member
to the team with the multi select seach user.
- If it is a unknown email, it will send an invitation,
- If it is a known user, it will add it to the team.
2024-03-25 18:08:44 +01:00
Anthony LC
904fae469d (app-desk) add useCreateTeamAccess react-query hook
Add the hook useCreateTeamAccess, it will be used to
add a member to a team.
2024-03-25 18:08:44 +01:00
Lebaud Antoine
da081b9887 🔧(renovate) open pull request with a default noChangeLog label
Discussed with @sampaccoud, Renovate PR do not necessitate ChangeLog updates
Our CI system requires a 'noChangeLog' label or an updated ChangeLog. Currently,
we manually add the label for each Renovate PR.

To improve DX, we configured Renovate to apply the label automatically.
2024-03-25 17:59:26 +01:00
Anthony LC
0bb5c0c5c2 ⬇️(app-desk) compatibility issue stylelint@16.3.0
Compatibility issue with stylelint@16.3.0 and
stylelint-prettier@5.0.0.
An issue has been opened in the `stylelint` repo.
2024-03-25 13:00:23 +01:00
renovate[bot]
a9bb556dfd ⬆️(dependencies) update js dependencies 2024-03-25 13:00:23 +01:00
renovate[bot]
32fa653c12 ⬆️(dependencies) update python dependencies 2024-03-25 08:54:42 +01:00
Anthony LC
7d9032b6ec 💚(app-desk) build template mail for e2e
The tests e2e were failing because the mail
template was not built.
We will use the job after the mail templates are build.
2024-03-22 17:26:32 +01:00
Anthony LC
897b68038f (app-desk) create invitation
Invite the selected members to the team.
To have a successful invitation:
- none user has this email
- an invitation is not pending for this email and
  this team
2024-03-22 17:26:32 +01:00
Anthony LC
bb9edd21da (app-desk) add useCreateInvitation react-query hook
Add the hook useCreateInvitation, it will be used to
create an invitation.
2024-03-22 17:26:32 +01:00
Anthony LC
bbf2695dee 🥅(app-desk) add data to APIError
We sometime need to get some data back when an
error occurs. We add a data prop to the APIError
class to allow us to do that.
2024-03-22 17:26:32 +01:00
Anthony LC
8b63807f38 ♻️(app-desk) create InputTeamName component
We created the InputTeamName component to use it
in all the places where we need to input the team name.
2024-03-22 14:53:40 +01:00
Anthony LC
88e38c4c7f 💄(app-desk) update ui
We update the theme of the app to match the design system:
- secondary button
- input error
- modal background
2024-03-22 14:53:40 +01:00
Anthony LC
8ea7b53286 (app-desk) modal update team
Integrate the modal and the logic to update a team.
2024-03-22 14:53:40 +01:00
Anthony LC
7347565f8d (app-desk) add useUpdateTeam react-query hook
Add the hook useUpdateTeam, it will be used to
update the name of a team.
2024-03-22 14:53:40 +01:00
Anthony LC
2d50920a48 ♻️(app-desk) create IconOptions component
Export from MemberAction component the icon options
to IconOptions component.
We will reuse this component in other places.
2024-03-22 14:53:40 +01:00
Anthony LC
36161972d7 ♻️(app-desk) keep team logic in teams feature
Part of the team logic was in the create team page,
we moved it to the CardCreateTeam component in
the teams feature.
It will be easier to maintain and reuse the logic.
2024-03-22 14:53:40 +01:00
Lebaud Antoine
f9fde490e8 🚀(smtp) update mail server configurations in staging
Update staging configuration, so they can use the outscale mail
gateway as recommended by @rouja.
2024-03-22 13:42:22 +01:00
Lebaud Antoine
1b3869c1e9 🌐(backend) generate traductions
With the recent addition of mails' templates, Django traduction files
needed to be updated.

It seems that recents backend changes were not reflected into the
Django traduction file. Fixed them, and add traductions related to
the invitation email.

Last revision was made on 2024-01-01
2024-03-22 13:42:22 +01:00
Lebaud Antoine
f6d5f737f4 💚(ci) download mails templates when testing back
build-mails job builds mails Django templates but was not persisting its
output. This steps was present in Joanie CI. It might have been removed,
when converting Circle CI worflows to Github Actions.

Artifacts are passed between build-mails and test-back jobs. test-back
job has now a dependency to  build-mails.
2024-03-22 13:42:22 +01:00
Lebaud Antoine
522914b47a (backend) email invitation to new users
When generating an Invitation object within the database, our intention
is to promptly notify the user via email. We send them an invitation
to join Desk.

This code is inspired by Joanie successful order flow.

Johann's design was missing a link to Desk, I simply added a button which
redirect to the staging url. This url is hardcoded, we should refactor it
when we will deploy Desk in pre-prod or prod environments.

Johann's design relied on Marianne font. I implemented a simpler version,
which uses a google font. That's not important for MVP.

Look and feel of this first invitation template is enough to make our PoC
functionnal, which is the more important.
2024-03-22 13:42:22 +01:00
Lebaud Antoine
1919dce3a9 🧑‍💻(views) render email's template
THis feature is inspired by Joanie. Add two new urls to render Emails
HTML and Text templates.

Developpers can render the email template they are working on. When necessary,
run make mails-build, and reload `_debug__/mail/hello_html`, it will re-render
the updated email template.

Also, I have copy/pasted one template extra tags from Joanie, which loads
bas64 string from static images. This code is necessary to render the dummy
template `hello.html`.
2024-03-22 13:42:22 +01:00
Lebaud Antoine
0141aa220f 🎨(models) extract invitation converter in a proper method
Improved code readability, by extracting this well-scoped unit of
logic in a dedicated method. Also, rename active_invitations to match
'valid' vocabulary used elsewhere in the doc. If no valid invitation
exists, early return to avoid nesting.
2024-03-22 13:31:24 +01:00
Anthony LC
159f112713 (app-desk) add role option to modal add members
When adding members to a team, the user can now
select the role of the members.
Only admin and owner can add new members to a team.
2024-03-22 11:13:24 +01:00
Anthony LC
faf699544b ♻️(app-desk) create ChooseRole component
We extract the ChooseRole component from the ModalRole
component to make it reusable.
2024-03-22 11:13:24 +01:00
Anthony LC
b8427d865f (app-desk) integrate multiselect search users
Integrate multiselect search users in the
modal add members.
We are using react-select to implement the
multiselect search users. We are using this
library in waiting for Cunningham to implement
the multiselect asynch component.
2024-03-22 11:13:24 +01:00
Anthony LC
a48dbde0ea 🧐(CI) add dummy data to test-e2e job
To search some users we need to have some
dummy data in the database.
This commit adds dummy data to the database
like users, teams, and identities.
2024-03-22 11:13:24 +01:00
Anthony LC
e9848bd199 (app-desk) add useUsers react-query hook
Add the hook useUsers, it will be used to
search users by name or email.
2024-03-22 11:13:24 +01:00
Anthony LC
1ad6ef8f96 🧑‍💻(frontend) remove CI control on traduction frontend
The CI was controlling if the traduction was made
in every PR. It makes the workflow quite grueling
when we have to change the literal, plus the synch
is complicating when we have multiple PR opened.

We remove the CI control on the traduction, we
will do dedicated PR to update the traduction.

We will add the CI control on the traduction in
the future, before a release by example.
2024-03-22 09:49:14 +01:00
Lebaud Antoine
97752e1d5f 🩹(factory) handle email uniqueness
When generating a batch of users using Faker, there's a possibility of
generating multiple users with the same email address. This breaches
the uniqueness constraint set on the email field, leading to flaky
tests that may fail when random behavior coincides unfavorably.

Implemented a method inspired by Identity's model to ensure unique
email addresses when creating user batches with Faker.
Updated relevant tests for improved stability.
2024-03-22 08:28:30 +01:00
Lebaud Antoine
99cee241f9 (api) support TeamAccess ordering on user-based fields
Important ordering fields for TeamAccess depend on user's
identities data. User and identities has a one-to-many relationship,
which forced us to prefetch the user-related data when listing
team's accesses.

Prefetch get data from the database using two SQL queries, and
join data in Python. User's data were not available in the first
SQL query.

Without annotating the query set with user main identities data,
we could not use default OrderingFilter backend code, which order_by()
the queryset.
2024-03-22 08:28:30 +01:00
Lebaud Antoine
6de0d013c3 (api) support TeamAccess ordering on their role
Enhance list capabilities, by adding the OrderingFilter as filter backend,
to the TeamAccess viewset.

API response can be ordered by TeamAccess role. More supported ordering
fields will be supported later on.
2024-03-22 08:28:30 +01:00
Lebaud Antoine
1de743e18a (pagination) add few tests on page's size
We created a custom Pagination class, were max_page_size is overriden.
I decided to add bare minimum tests to make sure we can set the maximum
page size using the 'page_size' query parameter.
2024-03-22 08:28:30 +01:00
Lebaud Antoine
756867da19 🔥(pagination) remove unused ordering field
Our Pagination class inherits from the PageNumberPagination Django class.
However, this base class as not ordering attribute. Thus, setting a
default value wont have any effect on the code.

Why did we end up passing a value to this non-existing attribute? Becasue
we copy/pasted some code sources from Joanie, and Joanie also has this
attribute set to a default value.

If you take a look at DRF pagination style documentation, the only three
attributes they set on the child class are 'page_size', 'max_page_size'
'page_size_query_param'. 'ordering' is not mentionned in the attributes
you may override. However, the CursorPagination class offers the latter
attribute, which may explain why we did end up setting this non-existing
attribute in Joanie.
2024-03-22 08:28:30 +01:00
Lebaud Antoine
d15adb4421 🐛(helm) fix wrongly named ingress
Admin ingress has been partially renamed to ingressAdmin.
I forgot to update helmfile values. Fixed them.
2024-03-21 17:51:09 +01:00
Marie PUPO JEAMMET
340ddf8b1a 🐛(dependencies) modify expected details on 404 responses
djangorestframework released version 3.15.0, which includes modifications of
details upon returning 404 errors (see related issue
https://github.com/encode/django-rest-framework/pull/8051).

This commit changes the expected details of 404 responses in our tests,
to match DRF 3.15.0.
2024-03-21 15:46:42 +01:00
renovate[bot]
2d0fb0ef70 ⬆️(dependencies) update python dependencies 2024-03-21 15:46:42 +01:00
Marie PUPO JEAMMET
7ef67037c3 (backend) convert invitations to accesses
Convert related invitations to accesses upon creating a new identity.
2024-03-21 12:14:10 +01:00
Anthony LC
f1124f6c09 🚸(app-desk) close modal role on click outside
The modal role will be closed when the user
clicks outside the modal.
The design does not have a close button, we removed it.
2024-03-21 11:13:17 +01:00
Anthony LC
2f8801f7eb (app-desk) add modal for adding members to a team
Create the button to open the modal.
Add a modal for adding members to a team.
This modal will open thanks to a dedicated page.
2024-03-21 11:13:17 +01:00
Anthony LC
4a141736ff 🎨(app-desk) add feature members
The feature teams is getting big, we extracted codes
related to members to a new feature members.
2024-03-21 11:13:17 +01:00
Lebaud Antoine
bdddbb84a5 📝(helm) update chart's README
Run the ./generate-readme.sh script to keep the README file
up to date with the values.yaml.
2024-03-21 10:49:58 +01:00
Lebaud Antoine
de4551ab30 🚀(helm) support Django Admin pages in ingress paths
Based on @rouja reco, I added a dedicated ingress to serve Django Admin
pages and Django statics. The admin route will be secured by the oauth proxy.

I simply copy/pasted the first ingress template, and adapted it.
2024-03-21 10:49:58 +01:00
Lebaud Antoine
e8a241adbc 🔧(helm) enable liveness and readiness probes on backend deployment
Enable the probes to track liveness and readiness of any backend pods.
Helm values were updated to enable the relevant configuration.
2024-03-21 10:49:58 +01:00
Lebaud Antoine
b3b1343796 🚀(helm) add a Redis cache service
This commit is working in progress. I have added an extra chart to take
benefits of the Redis operator developed by Indie hoster.

When using the dev environment, I used bitnami redis chart to deploy
a Redis service with authentication disable.
2024-03-21 10:49:58 +01:00
Lebaud Antoine
d49cc11ef1 🩹(helm) rename mismatching environment variable
CSRF trusted origins are set using an environment variable. The env
value was wrongly name to CORS_ALLOWED_ORIGINS, which doesn't exist
in our Django configurations. I fixed this minor issue.
2024-03-21 10:49:58 +01:00
Lebaud Antoine
28adf987f7 🔐(helm) add OIDC secrets for dev environment
Set OIDC secrets for the dev environment. Please note that we use different
secrets between dev and staging. Why? Benoit created two client id, thus we
could easily tests Agent Connect feature from the local host and the staging
one.

The local host is desk.127.0.0.1.nip.io. If this value change at any time,
please consider asking Benoit to update the host value linked to the dev
client id.
2024-03-21 10:49:58 +01:00
Jacques ROUSSEL
c6b8e47b29 🚀(helm) prepare staging deployment
Thx @rouja for your help on deploying Desk. This commit slightly modifies
helm charts and helmfile to prepare the initial project deployment in a
staging environment.

@rouja updates:
- added secrets files for dev and staging environments (dev's one is empty)
- disable ingress by default, to avoid any security issue
- added an extra chart to benefit from Indie hoster Postgres operator

Thx to this commit we deployed a first draft version figured out
that the Django session were broken. We are using a cache session engine,
and wrongly configure cache backend to local memory. Thus, Django server
is not able to resolve the session, and enters in an infinite loop to
log-in the user.
2024-03-21 10:49:58 +01:00
Lebaud Antoine
a8a001e1e4 🚀(helm) build a minimalistic dev Helmfile
Please note that this Helmfile is uncomplete, it lacks services as
redis, celery, mail ... which are declared in the Docker Compose file
but not yet used in development and production images.

Thus, to run the Desk Helm chart, we only add a postgres database to run the
Django backend server, and apply migrations.

For now, this Helmfile is quite hard to test in dev environment, because the
frontend redirects automatically to the SSO login page. We cannot really
assess if backend and frontend are working properly. We might adjust some
configurations after the first deployment in stagging.

(We are a bit in rush, to respect the current sprint deadline.)

Development values points https://desk.127.0.0.1.nip.io URL. Please note that
the frontend image for now has been built with this URL for the backend address.
Meaning that we either need to rebuild and publish a frontend image with the
staging URL when deploying the project, or enhance our frontend image, to pass
the backend URL at runtime.
2024-03-21 10:49:58 +01:00
Lebaud Antoine
bbd8e1b48d 🚀(helm) write desk Helm chart
First, thanks a LOT @rouja for your help along the way.
This commit propose a first draft of Helm chart to prepare deployment.
It follows Plane's Helm Chart, hosted on the shared team repo,
please https://github.com/numerique-gouv/helm-charts, PR #11

It offers advanced templating function under _helpers.tpl, an auto-generated
README file when running ./generate-readme.sh, and a clear files structure.

The chart itself is quite simple. We have two deployments, one for the
frontend and one for the backend. Both need a dedicated service, which are
exposed using a common ingress. Frontend is accessible from the / path and
backend's from /api path.

Please note, we added a backend job to migrate the database when deploying
backend's pods. This job should be auto-cleaning itself 100s after it completes
to avoid any error when syncing helm.

values.yaml file is quite pristine, all common env variables will be set
in helmfile configuration.

Deploying frontend static files through kubernetes is temporary, we plan to
either remplace it by an external CDN or use minio to host static output in
a S3 bucket within the cluster.
2024-03-21 10:49:58 +01:00
Anthony LC
f21966cca9 🌐(app-desk) order translations asc
When we pull the translations from crowdin we
get lot of git diff noise with the json file.
We order the keys in the json file to make the
diffs more readable.
2024-03-20 14:23:42 +01:00
Lebaud Antoine
e4a6b33366 🐛(docker) switch CMD form from Shell to Exec
`backend-development` and `backend-production` CMD syntaxes were
using a Shell Form. Shell form prevented Unix signals from reaching
our container correctly, such as SIGTERM. Also, the shell process
ends up being the PID 1, instead of our Python scripts.

Docker recommends to use the exec form whenever possible.
2024-03-20 09:31:19 +01:00
Lebaud Antoine
44b5999df8 🔧(backend) configure RedisCache in production settings
In development, sessions are saved in local memory. It's working well,
however it doesn't adapt to a kubernetized setup. Several pods need
to access the current sessions, which need to be stored in a single
source of truth.

With a local memory cache, pods cannot read session saved in other pods.
We end up returning 401 errors, because we cannot authenticate the user.

I preferred setting up a proper cache than storing sessions in database,
because in the long run it would be a performance bottleneck. Cache will
decrease data access latency when reading current sessions.

I added a Redis cache backend to the production settings. Sessions would
be persisted to Redis. In K8s, a Redis operator will make sure the cached
data are not lost.

Two new dependencies were added, redis and django-redis.

I followed the installation guide of django-redis dependency. These
setting were tested deploying the app to a local K8s cluster.
2024-03-19 16:57:27 +01:00
Anthony LC
f503120b3c 📌(frontend) pin @types/react-dom globally
Compatibility issues with `@types/react-dom`.
Force the usage of the same version of
`@types/react-dom` across all packages and
dependencies.
2024-03-18 14:07:17 +01:00
renovate[bot]
079968b532 ⬆️(dependencies) update js dependencies 2024-03-18 14:07:17 +01:00
Lebaud Antoine
8e76a0ee79 🔧(frontend) update production value for the API_URL env var
For now, the env variable should point to the only deployed environment,
staging. It'll allow @rouja deploying for the first time our project.
2024-03-15 16:32:58 +01:00
Lebaud Antoine
a2ff33663b 🚚(docker) make images naming consistent
It was quite confusing having development, production and
frontend images' names in the same Docker file. New comers
to the project would have some difficuluties when
differentiating frontend from backend images.

Try to make these naming more explicit and consistent.
Thanks @rouja for your recommendation.
2024-03-15 16:32:58 +01:00
Lebaud Antoine
78459df962 🐛(docker) build Docker images with an unprivileged user
This is a major issue. Docker Images were built and published with a
root user in the CI.

if a user manages to break out of the application running as root in the
container, he may gain root user access on host. In addition, configuring
container to user unprivileged is the best way yo prevent privilege
escalation attacks.

We mitigated this issue by creating a new environment variable DOCKER_USER.
DOCKER_USER is set with id -u and id -g outputs. Then, it is passed as a
build-args when running docker/build-push-action steps.
2024-03-15 16:32:58 +01:00
Lebaud Antoine
4579e668b6 ️(docker) add frontend dependencies to .dockerignore
Ignore frontend dependencies when coping frontend sources to build
the frontend Docker image. It would improve a bit performances locally,
when building the frontend image.
2024-03-15 16:32:58 +01:00
Lebaud Antoine
6ee39a01af 🎨(env) add missing newline at EOF
Found wrongly formatted files, fixed them.
2024-03-15 16:32:58 +01:00
Lebaud Antoine
3378d4b892 👷(frontend) push frontend image to DockerHub
Build and push the frontend image to DockerHub. Backend an Frontend
images will be stored in separate repos: people-backend and people-frontend.

It will be cleaner than managing all images in a single repo and creating
tags to discriminate frontend and backend images.

CI code is not factorized between jobs. Frontend and backend jobs could be
a bit factorized. Hovewer it might be a bit premature, and I prefer having
them decoupled for now. @rouja suggested to introduce a custom github actions
to avoid maintaining the same logic accross different repo.

Please not as the images are built from the same Dockerfile, it's important
to precise the right target.
2024-03-15 16:32:58 +01:00
Lebaud Antoine
c40f656622 ⬆️(project) upgrade mail-builder Node Image
Updated to Node Image version 20 to align with the frontend image. It will
save us having two different Node versions in the same docker file, and
should not impact mail-builder.
2024-03-15 16:32:58 +01:00
Lebaud Antoine
1a3b396230 (frontend) introduce frontend Docker Image
To facilitate deployment on Kubernetes, we've introduced a Docker image for the
frontend. The Next.js project is built, and its static output is served using an
Nginx reverse proxy.

Since DevOps lacks a certified cold storage solution (e.g., S3) for serving
static files, we've decided to containerize the frontend as a quick workaround
for deploying staging environments.

Please note this Docker Image is WIP. One of the main issue still not resolved
concerns environment variables, which are only available when building the
Docker Image. Thus, having different environment variables values between
environment (dev, pre-prod, prod) will require us to build several frontend
images, and tag them with the appropriate target environment.

The `.env.production` values are not the final ones. For now, they were set to
dev values. It allows us to test the frontend image with the development setup.

Important: The frontend image is built-on top of an unprivileged Nginx image,
which exposes by default port 8080 instead of 80 for classic Nginx image.
You can find more info https://github.com/nginxinc/docker-nginx-unprivileged.

The Docker Compose Nginx service is used to proxy OIDC requests to keycloak,
in order to share the same host when initiating an OIDC flow, from outside and
inside docker virtual network.

All Nginx configurations related to serve frontend static build were moved to a
newly created conf file under src/frontend/apps/desk. When starting the frontend
image, we desire to start the minimum Nignx config required to serve frontend
statics.
2024-03-15 16:32:58 +01:00
Samuel Paccoud - DINUM
759c06a289 🧑‍💻(demo) improve distribution in number of identities per user
The current implementation of our product demo via the make command lacks
user identity for a significant portion of generated users, limiting the
realism of the showcased scenarios. As it stands, users created by the make
command lack complete information, such as full names and email addresses,
because they don't have any identity.

I tried to come up with the simplest solution:
We now generate a very small portion of our users with 0 identities. The
probability for users to have only 1 identity is the highest but they
can have up to 4 with decreasing probabilities. I removed the possibility
to set a maximum number of identities as it doesn't bring any value.

3% percent of the identities created will have no email and 3% no name.

Fixes https://github.com/numerique-gouv/people/issues/90
2024-03-14 19:39:22 +01:00
Anthony LC
97d9714a0d 🐛(app-desk) close dropDown when click outside
When we were clicking outside the dropdown,
the dropdown was not closing.
This commit fixes that.
2024-03-14 09:14:25 +01:00
Anthony LC
c9e4d47d9d ️(frontend) clean yarn.lock
The yarn.lock file get full of garbage and old
dependencies after a while. This commit cleans it.
2024-03-13 11:31:50 +01:00
Anthony LC
b30bb6ce2f ♻️(app-desk) improve useCunninghamTheme
Some tokens were not available from the hook.

We only had the tokens of the currentTheme available
but actually the theme is an augmentation of the
default theme, so we should use the default theme
tokens as a base and then override them with the
currentTheme tokens.
It is what this commit does.
2024-03-13 11:31:50 +01:00
Anthony LC
8ae7b4e8e9 ♻️(app-desk) cunningham theme more dsfr
Mockup doesn't seem totally synch with DSFR design,
this commit will adapt buttons and input in a more
DSFR way.
2024-03-13 11:31:50 +01:00
Anthony LC
8b014e289a (app-desk) component BoxButton
We often need unstyled button to wrap around some content,
we were using Cunningham's button for this purpose,
but it is not the best choice as lot of style is applied
to their buttons.
This component is a simple wrapper around the button
element with all the Box functionalities. Usefull
for wrapping icons by example.
2024-03-13 11:31:50 +01:00
Lebaud Antoine
7d65de1938 (backend) search user on her email and name
Compute Trigram similarity on user's name, and sum it up
with existing one based on user's email.

This approach is inspired by Contact search feature, which
computes a Trigram similarity score on first name and last
name, to sum up their scores.

With a similarity score influenced by both email and name,
API results would reflect both email and name user's attributes.

As we sum up similarities, I increased the similarity threshold.
Its value is empirical, and was finetuned to avoid breaking
existing tests. Please note, the updated value is closer to the
threshold used to search contacts.

Email or Name can be None. Summing two similarity scores with
one of them None, results in a None total score. To mitigate
this issue, I added a default empty string value, to replace
None values. Thus, the similarity score on this default empty
string value is equal to 0 and not to None anymore.
2024-03-11 20:23:05 +01:00
Lebaud Antoine
b2d68df646 (backend) mock identities' name when searching a user
When testing user search, we generated few identities
with mocked emails.

Name attribute was introduced on Identity model. Currently
names are freely and randomly generated by the factory.

To make this mocked data more realist, mock also identities'
names to match their email.

It should not break existing tests, and will make them more
predictable when introducing advanced search features.
2024-03-11 20:23:05 +01:00
renovate[bot]
4f9f49ac9a ⬆️(dependencies) update js dependencies 2024-03-11 12:55:01 +01:00
renovate[bot]
421ef899da ⬆️(dependencies) update python dependencies 2024-03-11 12:25:23 +01:00
Lebaud Antoine
b416c57bbe 🩹(frontend) fix layout overflow in Team info
Few minor layout issues were fixed.

First display label and dates inline, so they wrap nicely
when screen's size decreases. It also fixes the text overflow
when the screen's size is tiny.

Then, align screen with the Figma design, where items are
justified on the left of the Team info component.
2024-03-11 12:17:17 +01:00
Marie PUPO JEAMMET
fa88f70cee 🐛(admin) prevent updating of invitations
Invitations cannot be updated for now. To reflect api behaviour,
we disable update in django admin as well.
2024-03-11 11:39:02 +01:00
Marie PUPO JEAMMET
b2956e42d3 🛂(abilities) fix anonymous and unrelated users accessing resources
The function computing abilities return "True" for method get,
even if role of request user was None.
2024-03-11 11:39:02 +01:00
Marie PUPO JEAMMET
18971a10e0 🚨(tests) fix back-end tests warnings
Fixes a warnings in back-end tests suite:
- post_generation hooks save
- ordering for invitation and user viewsets
2024-03-11 11:39:02 +01:00
Marie PUPO JEAMMET
62758763df (api) add invitations CRUD
Nest invitation router below team router and add create endpoints for
authenticated administrators/owners to invite new members to their team,
list valid and expired invitations or delete invite altogether.

Update will not be handled for now. Delete and recreate if needed.
2024-03-11 11:39:02 +01:00
Anthony LC
a15e46a2f9 🌐(app-desk) translate role in member grid
The roles in the member grid were not being translated.
This commit adds the translation for
the roles in the member grid.
2024-03-08 16:46:07 +01:00
Anthony LC
e15c7cb2f4 (app-desk) integrate modal to update roles
Integrate the design and functionality
for updating a member's role.
Managed use cases:
 - when the user is an admin
 - when the user is the last owner
 - when the user want to update other orner
2024-03-08 15:55:26 +01:00
Anthony LC
0648c2e8d3 (app-desk) integrate grid member action
Integrate the action button dropdown in
the member grid. For the moment it will be
used to update the role of a member.
Manage use cases:
 - Does not display when member's role
 - Does not display when member is an admin
   that wants to update owner role.
2024-03-08 15:55:26 +01:00
Anthony LC
b0d3f73ba2 🏷️(app-desk) update interfaces business logic
Update interfaces of User / Team / Access
to get what is needed for the frontend.
2024-03-08 15:55:26 +01:00
Anthony LC
9be973a776 (app-desk) add useUpdateTeamAccess react-query hook
Add the hook useUpdateTeamAccess, it will be used to
change the role of a member.
2024-03-08 15:55:26 +01:00
Anthony LC
150258b5a4 ⬆️(app-desk) upgrade Cunningham design system
The last version of the Cunningham design system
has some new features that we need in this
feature.
2024-03-08 15:55:26 +01:00
Anthony LC
b41fd1ab69 (app-desk) component DropButton
Button that opens a dropdown menu when clicked.
It will manage all the state related to
the dropdown menu.
Can be used controlled or uncontrolled.
2024-03-08 15:55:26 +01:00
Lebaud Antoine
b5ce19a28e 📝(backend) clarify how team accesses are queried
Break copy/pasted comment from Joanie in several inline
comments, that are more specific and easy to read.

Hopefully, it will help future myself understanding this
queryset and explaining it.
2024-03-07 19:55:53 +01:00
Lebaud Antoine
163f987132 🐛(backend) fix team accesses abilities
To compute accesses's abilities, we need to determine
which is the user's role in the team.

We opted for a subquery, which retrieves the user's role
within the team and annotate queryset's results.

The current subquery was broken, and retrieved other
users than the request's user. It led to compute accesses'
abilities based on a randomly picked user.
2024-03-07 19:55:53 +01:00
Lebaud Antoine
e9482a985f (backend) enhance tests when listing team accesses
Abilities on team accesses are computed based on request user role.
Thus, members' roles in relation with user's role matters a lot, to
ensure the abilities were correctly computed.

Complexified the test that lists team accesses while being authenticated.
More members are added to the team with privileged roles. The user
is added last to the less with the less privileged role, "member".

Order matters, because when computing the sub query to determine
user's role within the team, code use the first result value to set the
role to compute abilities.
2024-03-07 19:55:53 +01:00
Lebaud Antoine
43d802a73b 🎨(backend) early return in User factory
Avoid unnecessary nesting when code can early return.
Also, rename "item" to a more explicit name "user_entry".

it's very nit-picking, sorry.
2024-03-07 11:42:34 +01:00
Lebaud Antoine
b4e4940fd7 🚨(backend) update Ruff config to suppress deprecation warning
When running make ruff-check, a warning informs the user that
some config are deprecated, and gives her the step to migrate.

This warning appears after Ruff released its v0.2.0.
Fix it, by keeping our pyproject.toml up to date.
2024-03-07 11:31:31 +01:00
Lebaud Antoine
5ec0dcf206 🚨(backend) follow Ruff 2024.2 style introduced in v0.3.0
We recently updated Ruff from 0.2.2 to v0.3, which introduced
Ruff 2024.2 style. This new style updated Ruff formatter's behavior,
making our make lint command fails.

Ruff 2024.2 style add a blank line after the module docstring.
Please take a look at Ruff ChangeLog to get more info.
2024-03-07 11:31:31 +01:00
renovate[bot]
dad81c8d73 ⬆️(dependencies) update python dependencies 2024-03-07 11:31:31 +01:00
Anthony LC
b010a7b5a7 🤡(app-desk) remove mock endpoint teams accesses
The endpoint teams/accesses is ready.
We remove the mock and the related libraries
and use the real endpoint.
2024-03-04 17:52:52 +01:00
Anthony LC
e16f51ca20 (app-desk) integrate member list design
Integrate the member list design in the team page
based on the mockup.
2024-03-04 15:49:50 +01:00
Anthony LC
9d30bc88f1 🤡(app-desk) mock endpoint teams/:teamId/accesses/
We intercept the request to the endpoint teams/:teamId/accesses/
and return a json with dummy accesses of the team.
2024-03-04 15:49:50 +01:00
Anthony LC
1da978e121 (app-desk) add useTeamAccesses react-query hook
Add the hook useTeamAccesses, it queries the accesses
if a team. It is paginated.
2024-03-04 15:49:50 +01:00
Anthony LC
3bf8965209 🚚(app-desk) rename and group team Panel
- add folder Panel
- rename PanelTeams to TeamList
- rename PanelTeam to TeamItem
2024-03-04 15:49:50 +01:00
renovate[bot]
5b9d2cccc5 ⬆️(dependencies) update js dependencies 2024-03-04 15:16:15 +01:00
Marie PUPO JEAMMET
81243cfc9a (api) return user id, name and email on /team/<id>/accesses/
Add serializers to return basic user info when listing /team/<id>/accesses/
endpoint. This will allow front-end to retrieve members info without having
to query API for each user.id.
2024-03-03 23:00:05 +01:00
Marie PUPO JEAMMET
70b1b996df 🏗️(tests) separate team accesses tests by action
Small commit to separate team accesses tests into diferent files.
2024-03-03 23:00:05 +01:00
renovate[bot]
29d274ab7c ⬆️(dependencies) update python dependencies 2024-02-28 14:21:49 +01:00
Anthony LC
f17771fc9b (app-desk) fix error warning jest test logout
We had a error warning in the jest test logout with fetchApi,
window.location.replace had to be mocked to avoid the error.
2024-02-26 16:31:02 +01:00
Anthony LC
65e78cde68 ⬇️(app-desk) downgrade @openfun/cunningham-react
Downgrade @openfun/cunningham-react to 2.4.0, because of a
compatibility problem with Jest.

We add this package with this version to the ignore list
in renovate.json, when we will have a new compatible version, we will
remove it from the ignore list.
2024-02-26 16:31:02 +01:00
Anthony LC
33288ab225 ⬇️(app-desk) downgrade @types/react-dom
Downgrade @types/react-dom to 18.2.18.
The lastest version seems to have lot of compatibility
issues with other packages:
- @openfun/cunningham-react
- @tanstack/react-query-devtools
- next

We add this package with this version to the ignore list
in renovate.json, when we will have a new compatible version, we will
remove it from the ignore list.
2024-02-26 16:31:02 +01:00
renovate[bot]
a3c0069697 ⬆️(dependencies) update js dependencies 2024-02-26 16:31:02 +01:00
Anthony LC
b307b373bb (app-desk) add luxon to display date
Add luxon to display date in the team description.
The date are internationalized and formatted as the
mockup requested.
2024-02-25 20:48:51 +01:00
Anthony LC
f21740e5e5 👔(backend) add read fields to teams api
Some fields are missing for the frontend.
Add read fields to teams api:
- created_at
- updated_at
2024-02-25 20:48:51 +01:00
Anthony LC
035a7a1fcc 🏷️(app-desk) rename type TeamResponse to Team
Rename type TeamResponse to Team, the components
using this type don't need to know that the data
is coming from the API.
2024-02-25 20:48:51 +01:00
Anthony LC
3f7e5c88bc (app-desk) change backend settings for e2e tests
When we run e2e tests with the CI, we are doing lot of
calls to the backend in a short amount of time. This can
lead to a rate limit particulary on the "user/me" endpoint.
To avoid this, we will use different backend settings
for the e2e tests.
2024-02-25 20:31:27 +01:00
Anthony LC
51064ec236 🥅(app-desk) better error management
We don't know how the error body returned by the
api will be, so we handle it in a more generic way.
2024-02-25 20:31:27 +01:00
Anthony LC
195e738c3c 🚸(app-desk) add 404 page
- Add a 404 page.
- Redirect to 404 page when a team is not found.
2024-02-25 20:31:27 +01:00
Samuel Paccoud - DINUM
54497c1261 🔒️(settings) remove default value for setting OIDC_RP_CLIENT_SECRET
Secret settings should not contain any default value as we risk shipping
them to production. The default value can be set via an environment variable
in the `env.d/development/common` file: OIDC_RP_CLIENT_SECRET
2024-02-23 17:15:46 +01:00
Anthony LC
8d7c545d1a 🗃️(backend) add name field to identity
We need a name for the user when we display the members in the
frontend. This commit adds the name column to the identity model.
We sync the Keycloak user with the identity model when the user
logs in to fill and udpate the name automatically.
2024-02-23 17:15:46 +01:00
Anthony LC
8cbfb38cc4 🚚(app-desk) alias home with teams url path
In order the keep the url path consistent and correctly
structured, the homepage is aliased with the teams page.
2024-02-22 16:20:39 +01:00
Anthony LC
4bd8095975 🐛(i18n) dot in key was not added
The parser was not adding the dot in the key to the
json file sent to crowdin. Some translations were
not being translated correctly.
2024-02-22 14:28:04 +01:00
Anthony LC
fc8dc24ba2 (app-desk) add team info component
Add the team info component to the team page.
This component shows some informations about the team:
  - name
  - amount of members
  - date created
  - date updated
2024-02-22 14:28:04 +01:00
Anthony LC
95219a33b3 💄(app-desk) change the text color
The text color from the mockup is not blue but
a dark grey. This commit changes the base color of
the Text component to match the mockup.
2024-02-22 14:28:04 +01:00
Lebaud Antoine
26fbe9fbe7 ✏️(project) fix minor typos
Found typos and fixed them.
2024-02-22 11:59:36 +01:00
Lebaud Antoine
4cacfd3a45 ♻️(frontend) switch to Authorization Code flow
Instead of interacting with Keycloak, the frontend navigate to the
/authenticate endpoint, which starts the Authorization code flow.

When the flow is done, the backend redirect back to the SPA,
passing a session cookie and a csrf cookie.

Done:
- Query GET user/me to determine if user is authenticated yet
- Remove Keycloak js dependency, as all the OIDC logic is handled by the backend
- Store user's data instead of the JWT token
2024-02-22 11:59:36 +01:00
Lebaud Antoine
38c4d33791 (backend) support Authorization code flow
Integrate 'mozilla-django-oidc' dependency, to support
Authorization Code flow, which is required by Agent Connect.

Thus, we provide a secure back channel OIDC flow, and return
to the client only a session cookie.

Done:
- Replace JWT authentication by Session based authentication in DRF
- Update Django settings to make OIDC configurations easily editable
- Add 'mozilla-django-oidc' routes to our router
- Implement a custom Django Authentication class to adapt
'mozilla-django-oidc' to our needs

'mozilla-django-oidc' routes added are:
- /authenticate
- /callback (the redirect_uri called back by the Idp)
- /logout
2024-02-22 11:59:36 +01:00
Lebaud Antoine
ec28c28d47 (backend) drop JWT authentication in API tests
Force login to bypass authorization checks when necessary.

Note: Generating a session cookie through OIDC flow
is not supported while testing our API.
2024-02-22 11:59:36 +01:00
Lebaud Antoine
927d0e5a22 🔧(project) proxy Keycloak with nginx
Backend and Frontend send requests to Keycloak through Nginx.

Thus, all requests from frontend and backend shared a same host
when received by Keycloak.

Otherwise, the flow is initiated from http://localhost:8080. When the Backend
calls token endpoint from Keycloak container at http://keycloak:8080,
the JWT token issuer and sender are mismatching.
2024-02-22 11:59:36 +01:00
Lebaud Antoine
699854e76b 🔧(project) configure standard OIDC flow in Keycloak
Enforce Authorization Code flow, and disable Implicit flow.

Done:
- Rename client people-front to people
- Add a client secret shared with the backend
- Add allowed redirect uris
- Disable implicit flow and enable Authorization Code flow without PCKE
- Sign userinfo endpoint to return application/jwt content
2024-02-22 11:59:36 +01:00
Marie PUPO JEAMMET
63e059a4e6 🔥(backend) remove users systematic return of profile_contact
Custom UserManaged returned profile_contact field when returning users.
While this may be useful later, we'd currently rather have it return users.
2024-02-21 17:49:19 +01:00
Anthony LC
5113eb013b 💄(app-desk) highlight team selected
Highlight the selected team in the team list.
2024-02-20 16:25:31 +01:00
Anthony LC
45d05873e2 🔥(app-desk) remove FAQ part in header
The FAQ part in the header is not displayed in
the mockup anymore, we remove it.
2024-02-20 16:25:31 +01:00
Anthony LC
1f3ab759d7 ⬇️(app-desk) downgrade @types/react-dom
Downgrade @types/react-dom to 18.2.18.
The lastest version seems to have lot of compatibility
issues with other packages:
- @openfun/cunningham-react
- @tanstack/react-query-devtools
- next
2024-02-19 16:58:23 +01:00
renovate[bot]
8b5f5bf092 ⬆️(dependencies) update js dependencies 2024-02-19 16:58:23 +01:00
Anthony LC
cd2efbe40d ️(frontend) add stale cache
We add a stale cache to reduce the number of requests to
the server.
This will help to improve the performance of the application.
2024-02-19 15:22:09 +01:00
Anthony LC
8b0c20dbdc (app-desk) add team page
- add the team page, you can access to
the team page with the id of the team.
- Add link to the panel team to access to the
team page.
2024-02-19 15:22:09 +01:00
Anthony LC
d0562029e8 (app-desk) integrate team endpoint
Integrate team endpoint with react-query.
It will be used to get the team on the team page.
2024-02-19 15:22:09 +01:00
Anthony LC
77efb1a89c 💄(app-desk) do not count the owner in item list icon
In every team, the owner is always included in it,
so we shouldn't count the owner when we display the icon
to say if a team has members.
2024-02-19 15:04:53 +01:00
Anthony LC
4566e132e1 💄(app-desk) integrate the create team design
- Integrate the create team design based from the
mockup
- Manage the different states of the create team
2024-02-19 15:04:53 +01:00
Anthony LC
f818715a45 🚚(app-desk) change next router to pages
2 routers exists in Next.js, "app" router and "page" router.
The "app" router has a bug introduced in Next.js 13.4.14, which is
not fixed yet. For the moment we cannot use dynamic routes with
"app" router with an SPA. As advised by the Next.js team, we
migrated to the "pages" router.
2024-02-19 15:04:53 +01:00
renovate[bot]
7d90092020 ⬆️(dependencies) update python dependencies 2024-02-19 10:08:28 +01:00
Lebaud Antoine
469903c9eb ✏️(project) fix minor typos
Found typos, fixed them.
2024-02-16 16:03:52 +01:00
Lebaud Antoine
1f1253ab21 🔧(backend) activate container liveness probes
Enabled Dockerflow Django app by activating liveness probes. The previously
unavailable routes such as `__heartbeat__` and `__lbheartbeat__` are now
accessible. New endpoints include:
* GET /__version__
* GET /__heartbeat__
* GET /__lbheartbeat__
2024-02-16 15:16:30 +01:00
Lebaud Antoine
6620932371 🐛(project) run production image locally with docker-compose
The local deployment of the Production image through docker-compose was
failing due to issues in the Django configurations, influenced by Joanie.

The bug stemmed from a dependency on a development-specific package
(drf-spectacular-sidecar) while attempting to run the application in
production mode.

Changes Made:
- Introduced new Django settings for local demo environments.
- Uncommented the nginx configuration to address the production image
  deployment issues.
2024-02-16 15:16:30 +01:00
Marie PUPO JEAMMET
8e537d962c 🗃️(database) create invitation model
Create invitation model, factory and related tests to prepare back-end
for invitation endpoints. We chose to use a separate dedicated model
for separation of concerns, see
https://github.com/numerique-gouv/people/issues/25
2024-02-15 19:42:40 +01:00
Lebaud Antoine
6080af961a 🩹(backend) fix Django Admin UserAdmin page
*Broken Identity string representation
Resolving a format error in the Identity string representation caused by
potential None values in the email field. This issue was discovered when
attempting to access the User details page in the Django Admin

*Broken User creation form
The replacement of the User's username with an email led to errors in
the UserAdmin class. The base class used the 'username' field in the
'add_fieldsets' attribute. This problem was discovered while attempting to
create a new user in the Django Admin.
2024-02-15 15:54:07 +01:00
Anthony LC
aba376702f 🐛(app-desk) fix circular dependency problem
Some circular dependency problems started to appear with Jest.
This commit fixes the problem by removing the feature index
file and moving the exports to the respective feature.
2024-02-15 09:56:07 +01:00
Anthony LC
1e38174c1b ️(e2e) add workers to playwright with CI
We have added workers to playwright to run tests in parallel,
this will help us to run tests faster.
The tests run on a commun database, so to keep the tests
stable between browsers, we created 3 different
users to run the tests, it will avoid to have commun data
stepping on each other.
2024-02-15 09:56:07 +01:00
Anthony LC
0ab9f16cb3 🌐(app-desk) improve message missing translations
- Improve the message when a translation is missing in the app,
now it will show the key of the missing translation.
- It also show the translation that are in crowdin but not in the
app.

- Add missing translations
2024-02-15 09:56:07 +01:00
Anthony LC
c71463a385 🚚(app-desk) create route teams/create
- Create the routes teams/create.
- Add link to the buttons routing to this page
- Adapt design
2024-02-15 09:56:07 +01:00
Anthony LC
47ffa60a94 ️(app-desk) add infinite scrool to teams list
If we have a big list of teams, we need to add infinite
scroll to avoid loading all the teams at once.
2024-02-15 09:56:07 +01:00
Anthony LC
fafffd2391 (app-desk) add interaction to sort button
In the team panel, we have the possibility to sort the teams.
This commit adds the interaction to the button.
2024-02-15 09:56:07 +01:00
Anthony LC
36e2dc2378 ♻️(backend) api teams list ordering
Give the possibility to order the teams list by
creation date.
By default the list is ordered by
creation date descending.
2024-02-15 09:56:07 +01:00
Anthony LC
4f0465fd32 (app-desk) integrate teams panel design
- Integrate teams panel design based from the mockup.
- List teams from the API.
2024-02-15 09:56:07 +01:00
Anthony LC
148ea81aa9 🎨(app-desk) box default direction column
Change the default direction of the Box component to column
to have a behavior closer to a normal div.
2024-02-15 09:56:07 +01:00
renovate[bot]
9981b9c615 ⬆️(dependencies) update django to v5.0.2 [SECURITY] 2024-02-12 12:00:30 +01:00
Lebaud Antoine
d1cc1942dc 🩹(backend) enhance Django Admin for Team Slug
Make Team's Slug field non-editable in the Django admin. It avoid
UX issues by preventing accidental slug overwrites during updates.
The Slug is now displayed in the teams list view.
2024-02-12 11:47:46 +01:00
Lebaud Antoine
a7d72d0fab 👷(project) streamline Docker image publishing workflow
Refactored 'Hub' CI job for clarity, using 'docker/build-push-action.'
This dedicated workflow efficiently manages image releases on push tag
and main branch merges events.

'Hub' job was broken by Chat GPT translation from Circle CI.

Images are pushed to a temporary Docker Hub repository,
lasuite/people.

Duplicated 'build-docker' job was removed from people workflow.
2024-02-12 11:37:38 +01:00
Lebaud Antoine
46ad7435c8 🔐(project) add Docker Hub secrets
Added Docker Hub username and password, to shared secrets.
2024-02-12 11:37:38 +01:00
renovate[bot]
1d4d4ee902 ⬆️(dependencies) update python dependencies 2024-02-12 10:38:36 +01:00
Marie PUPO JEAMMET
c6ea7c0831 📝(api) use DRF constants instead of bare status codes
Replace bare status codes used in tests with DRF status code constants,
for more explicit tests.
2024-02-08 15:53:08 +01:00
Marie PUPO JEAMMET
d2bf44d2fd (models) add slug field to Team model
Add slug field for team objects. Unique slug based on team names,
in an effort to avoid duplicates.
2024-02-08 15:53:08 +01:00
Lebaud Antoine
c117f67952 ✏️(project) fix minor typos
Found typos and fixed them.
2024-02-06 08:58:21 +01:00
Anthony LC
ec2fcaa1dd ♻️(app-desk) create Auth component
- Create the Auth component, it will manage the authentication.
- Moved auth folder to features folder.
2024-02-05 16:08:22 +01:00
Anthony LC
0b703cda97 🚚(app-desk) move features to features folder
The features were in the app folder, app folder is where Next uses
his router system.
To avoid confusion between the folder router and the features,
we export the features in a feature folder.
2024-02-05 16:08:22 +01:00
Anthony LC
fc6487ddc1 🚚(app-desk) integrate next router with menu
Integrate next router with the menu.
Create most of the pages.
2024-02-05 16:08:22 +01:00
Anthony LC
66dbea3c6d (app-desk) integrate static menu in the app
Integrate the menu from the mockup in the app.
2024-02-05 16:08:22 +01:00
Anthony LC
cbe356214d 🚨(frontend) fix some eslint warnings
- Eslint tried to search some configs in the node_modules folder,
we ignore node_modules in the eslint config now.
- Adapt next eslint to use next/babel.
2024-02-05 16:08:22 +01:00
Anthony LC
ce55721b5d 💄(app-desk) handle svg as react component
The svg in nextjs was not handled as a react component, it was
not possible to change dynamically the color of the svg by example.
We add the @svgr/webpack, it will handle the svg as react component.
We keep as well the way next.js handle the svg, so we can use both
ways.
To handle the svg in the next way we need to add the
`?url` at the end of the svg import.
2024-02-05 16:08:22 +01:00
Anthony LC
32e42e126d 🌐(app-desk) translate Desk in french
Translate app Desk in french thanks to crowdin.
2024-02-05 15:34:37 +01:00
Anthony LC
8043d12315 🚨(i18n) add linter to i18n package
We need to add a linter to the i18n package, we are mainly
interested by the jest linting rules so we create
a jest eslint config pluggable our other configs and to the
i18n eslint config.
2024-02-05 15:34:37 +01:00
Anthony LC
801cb98e15 (i18n) install jest and add tests
We install Jest to test our i18n package.
We tests:
  - the extraction of the translations on the Desk app fo crowdin
  - the formatings of the translations from crowdin to the app
  - we check that all the translations are present in the app
We connect the tests to the CI.
2024-02-05 15:34:37 +01:00
Anthony LC
3d0824e023 🌐(i18n) create package i18n
We create a package i18n to manage the translations of the project.
It help us to extract the translations from the frontend to
be deployed to crowdin.
It also help us to format the translations from crowdin to
be used by the frontend apps.
2024-02-05 15:34:37 +01:00
Anthony LC
7add42f525 🌐(app-desk) plug i18n to LanguagePicker
- Plug i18n to LanguagePicker.
- Make translatable all the string of the app.
2024-02-05 15:34:37 +01:00
Anthony LC
01b7ad3f30 🌐(app-desk) install internationalization
Install internationalization in the Desk app.
We use react-i18next.
2024-02-05 15:34:37 +01:00
renovate[bot]
6a0ed04b0d ⬆️(dependencies) update python dependencies 2024-02-05 13:22:56 +01:00
Anthony LC
21550fe01d 🚨(frontend) fix linting issues
The prettier upgrade was causing some linting issues.
This commit fixes them.
2024-02-05 09:46:30 +01:00
renovate[bot]
92e3e11daf ⬆️(dependencies) update js dependencies 2024-02-05 09:46:30 +01:00
Lebaud Antoine
c54d457fa6 🎨(project) fixed minor code formatting issues in .sops.yaml
Identified and resolved minor code formatting issues.

Updates:
* Add a newline at the end of the file
* Capitalize first names and last names of all users.
2024-02-01 13:54:18 +01:00
Marie
6f18713a7e 📝(doc) improve test names and docstrings
improve test names and docstrings

Co-authored-by: aleb_the_flash <45729124+lebaudantoine@users.noreply.github.com>
2024-02-01 10:36:06 +01:00
Samuel Paccoud - DINUM
a4ac5304d7 🐛(api) return best matching identity only
Use best matching identity to order results.
2024-02-01 10:36:06 +01:00
Marie PUPO JEAMMET
3aba9a4419 🐛(api) enable search on identites instead of users
A previous PR enabled user search using the email. After discussion models,
we chose to enable research on identities, while still returning users.
2024-02-01 10:36:06 +01:00
Jacques ROUSSEL
5b0b2933a2 🔧(sops) update secrets
Decrypt and reencrypt secrets to grant access to marie's key
2024-01-31 18:50:58 +01:00
Marie PUPO JEAMMET
31a5518a5c 🔧(sops) add maries public key
add marie's key to grant her access
2024-01-31 18:50:58 +01:00
Anthony LC
13a58d6fa0 🚨(frontend) fix linting issues
The prettier upgrade was causing some linting issues.
This commit fixes them.
2024-01-30 09:26:33 +01:00
renovate[bot]
83d9310c26 ⬆️(dependencies) update js dependencies 2024-01-30 09:26:33 +01:00
renovate[bot]
6abcf98ad2 ⬆️(dependencies) update python dependencies
Fix new linter issues introduced by Ruff's upgrade.
2024-01-29 15:48:23 +01:00
Lebaud Antoine
ab7d466823 🔧(project) enable renovate dependency dashboard
Enable renovate bot to open an issue with a dependency dashboard.
2024-01-29 15:09:13 +01:00
Jacques ROUSSEL
ab9aac08b0 👷(ci) sops: Add age key
Add key for Antoine Lebaud
2024-01-29 14:39:37 +01:00
Lebaud Antoine
d5f16cddb0 🔧(project) update Renovate configuration to fix match manager issue
The current Renovate configuration is using the wrong match manager, as our
project utilizes `pyproject.toml` instead of `setup.cfg`. This commit corrects
the configuration to ensure compatibility with our project structure.
2024-01-29 13:24:17 +01:00
Jacques ROUSSEL
54f64838a0 👷(ci) sops: Add age key
Add key for Anthony Le-Courric
2024-01-29 12:10:49 +01:00
Marie
269ba42204 (api) search users by email (#16)
* (api) search users by email

The front end should be able to search users by email.
To that goal, we added a list method to the users viewset
thus creating the /users/ endpoint.
Results are filtered based on similarity with the query,
based on what preexisted for the /contacts/ endpoint.

* (api) test list users by email

Test search when complete, partial query,
accentuated and capital.
Also, lower similarity threshold for user search by email
as it was too high for some tests to pass.

* 💡(api) improve documentation and test comments

Improve user viewset documentation
and comments describing tests sections

Co-authored-by: aleb_the_flash <45729124+lebaudantoine@users.noreply.github.com>
Co-authored-by: Anthony LC <anthony.le-courric@mail.numerique.gouv.fr>

* 🛂(api) set isAuthenticated as base requirements

Instead of checking permissions or adding decorators
to every viewset, isAuthenticated is set as base requirement.

* 🛂(api) define throttle limits in settings

Use of Djando Rest Framework's throttle options, now set globally
to avoid duplicate code.

* 🩹(api) add email to user serializer

email field added to serializer. Tests modified accordingly.
I added the email field as "read only" to pass tests, but we need to discuss
that point in review.

* 🧱(api) move search logic to queryset

User viewset "list" method was overridden to allow search by email.
This removed the pagination. Instead of manually re-adding pagination at
the end of this method, I moved the search/filter logic to get_queryset,
to leave DRF handle pagination.

* (api) test throttle protection

Test that throttle protection succesfully blocks too many requests.

* 📝(tests) improve tests comment

Fix typos on comments and clarify which setting are tested on test_throttle test
(setting import required disabling pylint false positive error)

Co-authored-by: aleb_the_flash <45729124+lebaudantoine@users.noreply.github.com>

---------

Co-authored-by: aleb_the_flash <45729124+lebaudantoine@users.noreply.github.com>
Co-authored-by: Anthony LC <anthony.le-courric@mail.numerique.gouv.fr>
2024-01-29 10:14:17 +01:00
Jacques ROUSSEL
8f2f47d3b1 👷(ci) sops: configure workflows to use sops secrets
Github secrets are difficult to maintain in time because we do not have
a way to track them efficiently. So to avoid this issue, we prefer to use
sops encrypted files to manage our secrets.
2024-01-29 08:56:43 +01:00
Anthony LC
c2c6ae88db 🚨(frontend) create package eslint-config-people
We want to lint the e2e tests, we export the eslint config from the
app desk to a package in order to use it for the e2e tests and
for our apps.
2024-01-24 16:14:03 +01:00
Anthony LC
3b155b708c 🚨(app-desk) add eslint-plugin-jsx-a11y
eslint-plugin-jsx-a11y is a plugin that provides a set of
accessibility rules that can find common
accessibility problems in your React.js elements.
2024-01-24 16:14:03 +01:00
Anthony LC
e8186408a0 🚨(add-desk) remove hydratation warning
When working with Next, some browser (like Brave) gives us
warning about hydratation. This commit remove this warning.
2024-01-24 16:14:03 +01:00
Anthony LC
38997fef6a (app-desk) design static LanguagePicker
Design static LanguagePicker, we will add the interactivity later.
2024-01-24 16:14:03 +01:00
Anthony LC
5062cac623 (app-desk) design static Header
Design static Header, we will add the interactivity later.
2024-01-24 16:14:03 +01:00
Anthony LC
5b4fe1e77f 🧑‍💻(app-desk) add styled-components and create generic components
Add styled-components to the app-desk, it will help us to create
easily styled components.
We create 2 components, Box and Text, it is 2 generic components
that we help us to style quickly html elements. They use
the power of styled-components and Cunningham's design system.
2024-01-24 16:14:03 +01:00
Anthony LC
30721b9ef9 🏷️(app-desk) add custom-next.d.ts file
Add custom-next.d.ts file to augment types.
We add our environment variables to avoid mispelling
and to have better autocompletion.
2024-01-24 16:14:03 +01:00
Anthony LC
e2618d0e11 💄(app-desk) add DSFR theme to Cunningham
In order to have the look and feel of the DSFR, we
create a DSFR theme to the Cunningham design system.
2024-01-24 16:14:03 +01:00
Anthony LC
97e7d99c02 🏗️(project) expose app Desk to nginx
Now that we have a out folder for the Desk app, we can expose it
to our server nginx.
2024-01-23 12:59:15 +01:00
Anthony LC
9ee39e1068 🏗️(app-desk) export app in out folder
We export the app in the out folder. This is a static export,
so our app can be deployed and hosted
on any web server that can serve HTML/CSS/JS static assets.
2024-01-23 12:59:15 +01:00
Anthony LC
c6823ba698 🛂(app-desk) create fetchAPI
Create a fetch wrapper for the API calls, it will handle:
- add correct basename on the api request
- add Bearer automatically on the api request
- logout automatically on 401 request
2024-01-23 12:59:15 +01:00
Anthony LC
da851f508a 👷(CI) add test-e2e job to people.yml
Add test-e2e to people.yml, it will run e2e tests on every PR.
Steps:
  - set env vars for e2e tests
  - build and start docker servers
  (backend, keycloak, DB)
  - install playwright
  - build apps
  - run e2e tests
  - save reports
2024-01-23 12:59:15 +01:00
Anthony LC
5f280ae3fc (app-desk) e2e test app-desk
Tests:
- login to keycloak
- create new teams
- check teams are displayed
2024-01-23 12:59:15 +01:00
Anthony LC
2ef31a424a (project) install e2e playwright
Install playwright, adapt the config file and add a scripts to
run the tests.
e2e testing will monitor all our frontend applications,
so we install it in the frontend folder.
It configures the base of our monorepo.
2024-01-23 12:59:15 +01:00
Anthony LC
fc7747dddf 🚚(frontend) rename folder app to apps
The folder app will be used for more than one app, so it was
renamed to apps.
2024-01-23 12:59:15 +01:00
Anthony LC
ba21784eab (app-desk) add fetch-mock and adapt the tests
- Adapt the tests to use React-Query
- Install fetch-mock to mock the fetch requests with Jest.
- Create a test to show how to mock the fetch requests
with fetch-mock
2024-01-17 13:37:55 +01:00
Samuel Paccoud - DINUM
0c550ebd1c (demo) add a "demo" app to facilitate testing/working on the project
We designed it to allow creating a huge number of objects fast using
bulk creation.
2024-01-17 13:37:55 +01:00
Samuel Paccoud - DINUM
cfc35ac23e ♻️(models) rename *_on fields to *_at
This is only for display, nothing has changed in database.
2024-01-17 13:37:55 +01:00
Samuel Paccoud - DINUM
65cfe7ff2b (admin) configure a minimum admin with users and teams
We added identities and team accesses as inlines.
2024-01-17 13:37:55 +01:00
Samuel Paccoud - DINUM
8b026078bc (models) make user and authentication work with Keycloak and admin
The admin was broken as we did not worry about it up to now. On the frontend
we want to use OIDC authentication only but for the admin, it is better if
the default authentication works as well. To allow this, we propose to add
an "email" field to the user model and make it the identifier in place of
the usual username. Some changes are necessary to make the "createsuperuser"
management command work.

We also had to fix the "oidc_user_getter" method to make it work with Keycloak.
Some tests were added to secure that everything works as expected.
2024-01-17 13:37:55 +01:00
Anthony LC
e1688b923e 🧑‍💻(makefile) add command to interact with the front
Add command:
- install-front-desk
- start-front-desk-dev
2024-01-17 13:37:55 +01:00
Anthony LC
5aca2c48e3 (app-desk) create a basic feature Teams
As a prove of concept, to check the full process of our token,
we create a basic feature Teams.
This feature can create a team and list all teams.
We use react-query to manage the cache and the request to the API.
2024-01-17 13:37:55 +01:00
Anthony LC
d0b2f9c171 (app-desk) integrate keycloak
Integrate keycloak with the frontend.
We use the keycloak-js library to handle the
authentication and authorization.
We installed Zustand to handle the states of the
application.
We store the token and auth process in authStore.
2024-01-17 13:37:55 +01:00
Anthony LC
bf1b7736bb (keycloak) add keycloak as auth server
Keycloak is a open source identity and access management
for modern applications and services.
- add keycloak server in docker-compose
- add keycloak in frontend
2024-01-17 13:37:55 +01:00
Anthony LC
ae07bc9246 (app-desk) install jest
Jest is a JavaScript Testing Framework, usefull to test React
components and to do unit testing.
2024-01-16 14:26:07 +01:00
Anthony LC
05d9f6430d 🚨(app-desk) add css linter
eslint is not enough for css, so we need a css linter too.
This commit adds stylelint to the project, we configure it
with prettier and add a check during the build process.
2024-01-10 11:14:16 +01:00
Anthony LC
58f99545c0 👷(ci) github action job build-desk
Create the job build-desk in the workflow people.yml.
It will check that the app is linting and building correctly.
2024-01-10 11:14:16 +01:00
Anthony LC
f4ff27636d 💄(app-desk) integrate cunningham design system
Integrate cunningham design system into app desk.
It comes with some boilerplate code that will have to be
adapted to our needs when we will get the design.
2024-01-10 11:14:16 +01:00
Anthony LC
4999472005 🚀(app-desk) generate app Desk with nextjs
Generate the app with nextjs, includes:
- typescript
- eslint
- prettier
- css modules
2024-01-10 11:14:16 +01:00
Jacques ROUSSEL
e4b0ca86e5 👷(ci) fix ci issue with changelog on main
check-changelog should only runs on PR
2024-01-08 08:44:06 +01:00
Marie PUPO JEAMMET
7713225fc8 👷(ci) fix python linting
pylint and ruff weren't reporting linting issues
2024-01-08 08:42:13 +01:00
Jacques ROUSSEL
875b7cd866 👷(ci) fix ci issue with changelog
fix issue with changelog

Test
2024-01-05 17:44:19 +01:00
Samuel Paccoud - DINUM
b5a46eba33 👷(ci) fix CI running in github actions
The CI configuration file was translated from CircleCI to github
actions  a bit too fast and had not been tested yet.
2024-01-05 15:31:43 +01:00
Samuel Paccoud - DINUM
8ebfb8715d 🚨(pylint) make pylint work and fix issues found
Pylint was not installed and wrongly configured. After making
it work, we fix all the issues found so it can be added to our
CI requirements.
2024-01-05 15:31:43 +01:00
Samuel Paccoud - DINUM
eeec372957 (project) first proof of concept based of Joanie
Used https://github.com/openfun/joanie as boilerplate, ran a few
transformations with ChapGPT  and adapted models and endpoints to
fit to my current vision of the project.
2024-01-03 16:31:08 +01:00
989 changed files with 28328 additions and 88524 deletions

View File

@@ -32,6 +32,5 @@ db.sqlite3
.pylint.d
.pytest_cache
# Frontend
# Frontend dependencies
node_modules
.next

23
.gitattributes vendored
View File

@@ -1,23 +0,0 @@
# Set the default behavior for all files
* text=auto eol=lf
# Binary files (should not be modified)
*.png binary
*.jpg binary
*.jpeg binary
*.gif binary
*.ico binary
*.mov binary
*.mp4 binary
*.mp3 binary
*.flv binary
*.fla binary
*.swf binary
*.gz binary
*.zip binary
*.7z binary
*.ttf binary
*.woff binary
*.woff2 binary
*.eot binary
*.pdf binary

View File

@@ -1,7 +1,7 @@
---
name: 🐛 Bug Report
about: If something is not working as expected 🤔.
labels: ["bug", "triage"]
---
## Bug Report
@@ -18,8 +18,8 @@ A clear and concise description of what you expected to happen (or code).
3. And then the bug happens!
**Environment**
- Docs version:
- Instance url:
- People version:
- Platform:
**Possible Solution**
<!--- Only if you have suggestions on a fix for the bug -->

View File

@@ -1,7 +1,7 @@
---
name: ✨ Feature Request
about: I have a suggestion (and may want to build it 💪)!
labels: ["feature", "triage"]
---
## Feature Request
@@ -16,8 +16,8 @@ A clear and concise description of what you want to happen. Add any considered d
A clear and concise description of any alternative solutions or features you've considered.
**Discovery, Documentation, Adoption, Migration Strategy**
If you can, explain how users will be able to use this and possibly write out some documentation (if applicable).
Maybe add a screenshot or design?
If you can, explain how users will be able to use this and possibly write out a version the docs (if applicable).
Maybe a screenshot or design?
**Do you want to work on it through a Pull Request?**
<!-- Make sure to coordinate with us before you spend too much time working on an implementation! -->

View File

@@ -1,13 +1,17 @@
---
name: 🤗 Support Question
about: If you have a question 💬, or something was not clear from the docs!
labels: ["support", "triage"]
---
## Support request
**Checks before filing**
Please make sure you have read our [main Readme](https://github.com/suitenumerique/docs).
Also make sure it was not already answered in [an open or close issue](https://github.com/suitenumerique/docs/issues?q=is%3Aissue%20state%3Aopen%20label%3Asupport).
---
<!-- ^ Click "Preview" for a nicer view! ^
We primarily use GitHub as an issue tracker. If however you're encountering an issue not covered in the docs, we may be able to help! -->
---
Please make sure you have read our [main Readme](https://github.com/numerique-gouv/people).
Also make sure it was not already answered in [an open or close issue](https://github.com/numerique-gouv/people/issues).
If your question was not covered, and you feel like it should be, fire away! We'd love to improve our docs! 👌

View File

@@ -1,22 +1,11 @@
## Purpose
Describe the purpose of this pull request.
Description...
## Proposal
- [ ] item 1...
- [ ] item 2...
Description...
## External contributions
Thank you for your contribution! 🎉
Please ensure the following items are checked before submitting your pull request:
- [ ] I have read and followed the [contributing guidelines](https://github.com/suitenumerique/docs/blob/main/CONTRIBUTING.md)
- [ ] I have read and agreed to the [Code of Conduct](https://github.com/suitenumerique/docs/blob/main/CODE_OF_CONDUCT.md)
- [ ] I have signed off my commits with `git commit --signoff` (DCO compliance)
- [ ] I have signed my commits with my SSH or GPG key (`git commit -S`)
- [ ] My commit messages follow the required format: `<gitmoji>(type) title description`
- [ ] I have added a changelog entry under `## [Unreleased]` section (if noticeable change)
- [ ] I have added corresponding tests for new features or bug fixes (if applicable)
- [] item 1...
- [] item 2...

View File

@@ -1,77 +0,0 @@
name: Download translations from Crowdin
on:
workflow_dispatch:
push:
branches:
- 'release/**'
jobs:
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
node_version: '22.x'
with-front-dependencies-installation: true
synchronize-with-crowdin:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Create empty source files
run: |
touch src/backend/locale/django.pot
mkdir -p src/frontend/packages/i18n/locales/impress/
touch src/frontend/packages/i18n/locales/impress/translations-crowdin.json
# crowdin workflow
- name: crowdin action
uses: crowdin/github-action@v2
with:
config: crowdin/config.yml
upload_sources: false
upload_translations: false
download_translations: true
create_pull_request: false
push_translations: false
push_sources: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# A numeric ID, found at https://crowdin.com/project/<projectName>/tools/api
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
# Visit https://crowdin.com/settings#api-key to create this token
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
CROWDIN_BASE_PATH: "../src/"
# frontend i18n
- name: Restore the frontend cache
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: generate translations files
working-directory: src/frontend
run: yarn i18n:deploy
# Create a new PR
- name: Create a new Pull Request with new translated strings
uses: peter-evans/create-pull-request@v7
with:
commit-message: |
🌐(i18n) update translated strings
Update translated files with new translations
title: 🌐(i18n) update translated strings
body: |
## Purpose
update translated strings
## Proposal
- [x] update translated strings
branch: i18n/update-translations
labels: i18n

View File

@@ -1,76 +0,0 @@
name: Update crowdin sources
on:
workflow_dispatch:
push:
branches:
- main
jobs:
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
node_version: '22.x'
with-front-dependencies-installation: true
with-build_mails: true
synchronize-with-crowdin:
needs: install-dependencies
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
# Backend i18n
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: "3.13.3"
- name: Upgrade pip and setuptools
run: pip install --upgrade pip setuptools
- name: Install development dependencies
run: pip install --user .
working-directory: src/backend
- name: Restore the mail templates
uses: actions/cache@v4
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
fail-on-cache-miss: true
- name: Install gettext
run: |
sudo apt-get update
sudo apt-get install -y gettext pandoc
- name: generate pot files
working-directory: src/backend
run: |
DJANGO_CONFIGURATION=Build python manage.py makemessages -a --keep-pot
# frontend i18n
- name: Restore the frontend cache
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: generate source translation file
working-directory: src/frontend
run: yarn i18n:extract
# crowdin workflow
- name: crowdin action
uses: crowdin/github-action@v2
with:
config: crowdin/config.yml
upload_sources: true
upload_translations: false
download_translations: false
create_pull_request: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# A numeric ID, found at https://crowdin.com/project/<projectName>/tools/api
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
# Visit https://crowdin.com/settings#api-key to create this token
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
CROWDIN_BASE_PATH: "../src/"

View File

@@ -1,85 +0,0 @@
name: Dependency reusable workflow
on:
workflow_call:
inputs:
node_version:
required: false
default: '22.x'
type: string
with-front-dependencies-installation:
type: boolean
default: false
with-build_mails:
type: boolean
default: false
jobs:
front-dependencies-installation:
if: ${{ inputs.with-front-dependencies-installation == true }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Setup Node.js
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node_version }}
- name: Install dependencies
if: steps.front-node_modules.outputs.cache-hit != 'true'
run: cd src/frontend/ && yarn install --frozen-lockfile
- name: Cache install frontend
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
build-mails:
if: ${{ inputs.with-build_mails == true }}
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/mail
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore the mail templates
uses: actions/cache@v4
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
- name: Setup Node.js
if: steps.mail-templates.outputs.cache-hit != 'true'
uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node_version }}
- name: Install yarn
if: steps.mail-templates.outputs.cache-hit != 'true'
run: npm install -g yarn
- name: Install node dependencies
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn install --frozen-lockfile
- name: Build mails
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn build
- name: Cache mail templates
if: steps.mail-templates.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}

52
.github/workflows/deploy.yml vendored Normal file
View File

@@ -0,0 +1,52 @@
name: Deploy
on:
push:
tags:
- 'preprod'
- 'production'
jobs:
notify-argocd:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "people,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/people/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Call argocd github webhook
run: |
data='{"ref": "'$GITHUB_REF'","repository": {"html_url":"'$GITHUB_SERVER_URL'/'$GITHUB_REPOSITORY'"}}'
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac ''${ARGOCD_WEBHOOK_SECRET}'' | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" $ARGOCD_WEBHOOK_URL
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac ''${ARGOCD_PRODUCTION_WEBHOOK_SECRET}'' | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" $ARGOCD_PRODUCTION_WEBHOOK_URL
start-test-on-preprod:
needs:
- notify-argocd
runs-on: ubuntu-latest
if: startsWith(github.event.ref, 'refs/tags/preprod')
steps:
-
name: Debug
run: |
echo "Start test when preprod is ready"

View File

@@ -1,18 +1,15 @@
name: Docker Hub Workflow
run-name: Docker Hub Workflow
on:
workflow_dispatch:
push:
branches:
- 'main'
- 'feature/doc-dnd'
tags:
- 'v*'
pull_request:
branches:
- 'main'
- 'ci/trivy-fails'
env:
DOCKER_USER: 1001:127
@@ -21,28 +18,39 @@ jobs:
build-and-push-backend:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "people,secrets"
-
name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: lasuite/impress-backend
images: lasuite/people-backend
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/people/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
run: echo "${{ secrets.DOCKER_HUB_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_HUB_USER }}" --password-stdin
-
name: Run trivy scan
uses: numerique-gouv/action-trivy-cache@main
with:
docker-build-args: '--target backend-production -f Dockerfile'
docker-image-name: 'docker.io/lasuite/impress-backend:${{ github.sha }}'
run: echo "$DOCKER_HUB_PASSWORD" | docker login -u "$DOCKER_HUB_USER" --password-stdin
-
name: Build and push
uses: docker/build-push-action@v6
uses: docker/build-push-action@v5
with:
context: .
target: backend-production
@@ -54,68 +62,42 @@ jobs:
build-and-push-frontend:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "people,secrets"
-
name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: lasuite/impress-frontend
images: lasuite/people-frontend
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/people/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
run: echo "${{ secrets.DOCKER_HUB_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_HUB_USER }}" --password-stdin
-
name: Run trivy scan
uses: numerique-gouv/action-trivy-cache@main
with:
docker-build-args: '-f src/frontend/Dockerfile --target frontend-production'
docker-image-name: 'docker.io/lasuite/impress-frontend:${{ github.sha }}'
run: echo "$DOCKER_HUB_PASSWORD" | docker login -u "$DOCKER_HUB_USER" --password-stdin
-
name: Build and push
uses: docker/build-push-action@v6
uses: docker/build-push-action@v5
with:
context: .
file: ./src/frontend/Dockerfile
target: frontend-production
build-args: |
DOCKER_USER=${{ env.DOCKER_USER }}:-1000
PUBLISH_AS_MIT=false
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-y-provider:
runs-on: ubuntu-latest
steps:
-
name: Checkout repository
uses: actions/checkout@v4
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: lasuite/impress-y-provider
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
run: echo "${{ secrets.DOCKER_HUB_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_HUB_USER }}" --password-stdin
-
name: Run trivy scan
uses: numerique-gouv/action-trivy-cache@main
with:
docker-build-args: '-f src/frontend/servers/y-provider/Dockerfile --target y-provider'
docker-image-name: 'docker.io/lasuite/impress-y-provider:${{ github.sha }}'
-
name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: ./src/frontend/servers/y-provider/Dockerfile
target: y-provider
build-args: DOCKER_USER=${{ env.DOCKER_USER }}:-1000
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
@@ -128,9 +110,29 @@ jobs:
runs-on: ubuntu-latest
if: github.event_name != 'pull_request'
steps:
- uses: numerique-gouv/action-argocd-webhook-notification@main
id: notify
-
uses: actions/create-github-app-token@v1
id: app-token
with:
deployment_repo_path: "${{ secrets.DEPLOYMENT_REPO_URL }}"
argocd_webhook_secret: "${{ secrets.ARGOCD_PREPROD_WEBHOOK_SECRET }}"
argocd_url: "${{ vars.ARGOCD_PREPROD_WEBHOOK_URL }}"
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "people,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/people/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Call argocd github webhook
run: |
data='{"ref": "'$GITHUB_REF'","repository": {"html_url":"'$GITHUB_SERVER_URL'/'$GITHUB_REPOSITORY'"}}'
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac ''${ARGOCD_WEBHOOK_SECRET}'' | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" $ARGOCD_WEBHOOK_URL

View File

@@ -1,30 +0,0 @@
name: Helmfile lint
run-name: Helmfile lint
on:
push:
pull_request:
branches:
- 'main'
jobs:
helmfile-lint:
runs-on: ubuntu-latest
container:
image: ghcr.io/helmfile/helmfile:v0.171.0
steps:
-
name: Checkout repository
uses: actions/checkout@v4
-
name: Helmfile lint
shell: bash
run: |
set -e
HELMFILE=src/helm/helmfile.yaml
environments=$(awk 'BEGIN {in_env=0} /^environments:/ {in_env=1; next} /^---/ {in_env=0} in_env && /^ [^ ]/ {gsub(/^ /,""); gsub(/:.*$/,""); print}' "$HELMFILE")
for env in $environments; do
echo "################### $env lint ###################"
helmfile -e $env -f $HELMFILE lint || exit 1
echo -e "\n"
done

View File

@@ -1,138 +0,0 @@
name: Frontend Workflow
on:
push:
branches:
- main
pull_request:
branches:
- "*"
jobs:
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
node_version: '22.x'
with-front-dependencies-installation: true
test-front:
needs: install-dependencies
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "22.x"
- name: Restore the frontend cache
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: Test App
run: cd src/frontend/ && yarn test
lint-front:
runs-on: ubuntu-latest
needs: install-dependencies
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "22.x"
- name: Restore the frontend cache
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: Check linting
run: cd src/frontend/ && yarn lint
test-e2e-chromium:
runs-on: ubuntu-latest
needs: install-dependencies
timeout-minutes: 20
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "22.x"
- name: Restore the frontend cache
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: Set e2e env variables
run: cat env.d/development/common.e2e.dist >> env.d/development/common.dist
- name: Install Playwright Browsers
run: cd src/frontend/apps/e2e && yarn install --frozen-lockfile && yarn install-playwright chromium
- name: Start Docker services
run: make bootstrap-e2e FLUSH_ARGS='--no-input'
- name: Run e2e tests
run: cd src/frontend/ && yarn e2e:test --project='chromium'
- uses: actions/upload-artifact@v4
if: always()
with:
name: playwright-chromium-report
path: src/frontend/apps/e2e/report/
retention-days: 7
test-e2e-other-browser:
runs-on: ubuntu-latest
needs: test-e2e-chromium
timeout-minutes: 20
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "22.x"
- name: Restore the frontend cache
uses: actions/cache@v4
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: Set e2e env variables
run: cat env.d/development/common.e2e.dist >> env.d/development/common.dist
- name: Install Playwright Browsers
run: cd src/frontend/apps/e2e && yarn install --frozen-lockfile && yarn install-playwright firefox webkit chromium
- name: Start Docker services
run: make bootstrap-e2e FLUSH_ARGS='--no-input'
- name: Run e2e tests
run: cd src/frontend/ && yarn e2e:test --project=firefox --project=webkit
- uses: actions/upload-artifact@v4
if: always()
with:
name: playwright-other-report
path: src/frontend/apps/e2e/report/
retention-days: 7

View File

@@ -1,204 +0,0 @@
name: Main Workflow
on:
push:
branches:
- main
pull_request:
branches:
- "*"
jobs:
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
with-build_mails: true
lint-git:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request' # Makes sense only for pull requests
steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: show
run: git log
- name: Enforce absence of print statements in code
run: |
! git diff origin/${{ github.event.pull_request.base.ref }}..HEAD -- . ':(exclude)**/impress.yml' | grep "print("
- name: Check absence of fixup commits
run: |
! git log | grep 'fixup!'
- name: Install gitlint
run: pip install --user requests gitlint
- name: Lint commit messages added to main
run: ~/.local/bin/gitlint --commits origin/${{ github.event.pull_request.base.ref }}..HEAD
check-changelog:
runs-on: ubuntu-latest
if: |
contains(github.event.pull_request.labels.*.name, 'noChangeLog') == false &&
github.event_name == 'pull_request'
steps:
- name: Checkout repository
uses: actions/checkout@v3
with:
fetch-depth: 50
- name: Check that the CHANGELOG has been modified in the current branch
run: git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.event.after }} | grep 'CHANGELOG.md'
lint-changelog:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Check CHANGELOG max line length
run: |
max_line_length=$(cat CHANGELOG.md | grep -Ev "^\[.*\]: https://github.com" | wc -L)
if [ $max_line_length -ge 80 ]; then
echo "ERROR: CHANGELOG has lines longer than 80 characters."
exit 1
fi
lint-spell-mistakes:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Install codespell
run: pip install --user codespell
- name: Check for typos
run: |
codespell \
--check-filenames \
--ignore-words-list "Dokument,afterAll,excpt,statics" \
--skip "./git/" \
--skip "**/*.po" \
--skip "**/*.pot" \
--skip "**/*.json" \
--skip "**/yarn.lock"
lint-back:
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/backend
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: "3.13.3"
- name: Upgrade pip and setuptools
run: pip install --upgrade pip setuptools
- name: Install development dependencies
run: pip install --user .[dev]
- name: Check code formatting with ruff
run: ~/.local/bin/ruff format . --diff
- name: Lint code with ruff
run: ~/.local/bin/ruff check .
- name: Lint code with pylint
run: ~/.local/bin/pylint .
test-back:
runs-on: ubuntu-latest
needs: install-dependencies
defaults:
run:
working-directory: src/backend
services:
postgres:
image: postgres:16
env:
POSTGRES_DB: impress
POSTGRES_USER: dinum
POSTGRES_PASSWORD: pass
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
env:
DJANGO_CONFIGURATION: Test
DJANGO_SETTINGS_MODULE: impress.settings
DJANGO_SECRET_KEY: ThisIsAnExampleKeyForTestPurposeOnly
OIDC_OP_JWKS_ENDPOINT: /endpoint-for-test-purpose-only
DB_HOST: localhost
DB_NAME: impress
DB_USER: dinum
DB_PASSWORD: pass
DB_PORT: 5432
STORAGES_STATICFILES_BACKEND: django.contrib.staticfiles.storage.StaticFilesStorage
AWS_S3_ENDPOINT_URL: http://localhost:9000
AWS_S3_ACCESS_KEY_ID: impress
AWS_S3_SECRET_ACCESS_KEY: password
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Create writable /data
run: |
sudo mkdir -p /data/media && \
sudo mkdir -p /data/static
- name: Restore the mail templates
uses: actions/cache@v4
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
fail-on-cache-miss: true
- name: Start MinIO
run: |
docker pull minio/minio
docker run -d --name minio \
-p 9000:9000 \
-e "MINIO_ACCESS_KEY=impress" \
-e "MINIO_SECRET_KEY=password" \
-v /data/media:/data \
minio/minio server --console-address :9001 /data
# Tool to wait for a service to be ready
- name: Install Dockerize
run: |
curl -sSL https://github.com/jwilder/dockerize/releases/download/v0.8.0/dockerize-linux-amd64-v0.8.0.tar.gz | sudo tar -C /usr/local/bin -xzv
- name: Wait for MinIO to be ready
run: |
dockerize -wait tcp://localhost:9000 -timeout 10s
- name: Configure MinIO
run: |
MINIO=$(docker ps | grep minio/minio | sed -E 's/.*\s+([a-zA-Z0-9_-]+)$/\1/')
docker exec ${MINIO} sh -c \
"mc alias set impress http://localhost:9000 impress password && \
mc alias ls && \
mc mb impress/impress-media-storage && \
mc version enable impress/impress-media-storage"
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: "3.13.3"
- name: Install development dependencies
run: pip install --user .[dev]
- name: Install gettext (required to compile messages) and MIME support
run: |
sudo apt-get update
sudo apt-get install -y gettext pandoc shared-mime-info
sudo wget https://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/conf/mime.types -O /etc/mime.types
- name: Generate a MO file from strings extracted from the project
run: python manage.py compilemessages
- name: Run tests
run: ~/.local/bin/pytest -n 2

384
.github/workflows/people.yml vendored Normal file
View File

@@ -0,0 +1,384 @@
name: People Workflow
on:
push:
branches:
- main
pull_request:
branches:
- '*'
jobs:
lint-git:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request' # Makes sense only for pull requests
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: show
run: git log
- name: Enforce absence of print statements in code
run: |
! git diff origin/${{ github.event.pull_request.base.ref }}..HEAD -- . ':(exclude)**/people.yml' | grep "print("
- name: Check absence of fixup commits
run: |
! git log | grep 'fixup!'
- name: Install gitlint
run: pip install --user requests gitlint
- name: Lint commit messages added to main
run: ~/.local/bin/gitlint --commits origin/${{ github.event.pull_request.base.ref }}..HEAD
check-changelog:
runs-on: ubuntu-latest
if: |
contains(github.event.pull_request.labels.*.name, 'noChangeLog') == false &&
github.event_name == 'pull_request'
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check that the CHANGELOG has been modified in the current branch
run: git whatchanged --name-only --pretty="" origin/${{ github.event.pull_request.base.ref }}..HEAD | grep CHANGELOG
lint-changelog:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Check CHANGELOG max line length
run: |
max_line_length=$(cat CHANGELOG.md | grep -Ev "^\[.*\]: https://github.com" | wc -L)
if [ $max_line_length -ge 80 ]; then
echo "ERROR: CHANGELOG has lines longer than 80 characters."
exit 1
fi
install-front:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '18.x'
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Install dependencies
if: steps.front-node_modules.outputs.cache-hit != 'true'
run: cd src/frontend/ && yarn install --frozen-lockfile
- name: Cache install frontend
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
build-front:
runs-on: ubuntu-latest
needs: install-front
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Build CI App
run: cd src/frontend/ && yarn ci:build
- name: Cache build frontend
uses: actions/cache@v4
with:
path: src/frontend/apps/desk/out/
key: build-front-${{ github.run_id }}
test-front:
runs-on: ubuntu-latest
needs: install-front
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Test App
run: cd src/frontend/ && yarn app:test
lint-front:
runs-on: ubuntu-latest
needs: install-front
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Check linting
run: cd src/frontend/ && yarn lint
test-e2e:
runs-on: ubuntu-latest
needs: [build-mails, build-front]
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set services env variables
run: |
make create-env-files
cat env.d/development/common.e2e.dist >> env.d/development/common
- name: Download mails' templates
uses: actions/download-artifact@v4
with:
name: mails-templates
path: src/backend/core/templates/mail
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Restore the build cache
uses: actions/cache@v4
id: cache-build
with:
path: src/frontend/apps/desk/out/
key: build-front-${{ github.run_id }}
- name: Build and Start Docker Servers
env:
DOCKER_BUILDKIT: 1
COMPOSE_DOCKER_CLI_BUILD: 1
run: |
docker-compose build --pull --build-arg BUILDKIT_INLINE_CACHE=1
make run
- name: Apply DRF migrations
run: |
make migrate
- name: Add dummy data
run: |
make demo FLUSH_ARGS='--no-input'
- name: Install Playwright Browsers
run: cd src/frontend/apps/e2e && yarn install
- name: Run e2e tests
run: cd src/frontend/ && yarn e2e:test
- uses: actions/upload-artifact@v4
if: always()
with:
name: playwright-report
path: src/frontend/apps/e2e/report/
retention-days: 7
build-mails:
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/mail
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: '18'
- name: Install yarn
run: npm install -g yarn
- name: Install node dependencies
run: yarn install --frozen-lockfile
- name: Build mails
run: yarn build
- name: Persist mails' templates
uses: actions/upload-artifact@v4
with:
name: mails-templates
path: src/backend/core/templates/mail
lint-back:
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/backend
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install development dependencies
run: pip install --user .[dev]
- name: Check code formatting with ruff
run: ~/.local/bin/ruff format . --diff
- name: Lint code with ruff
run: ~/.local/bin/ruff check .
- name: Lint code with pylint
run: ~/.local/bin/pylint .
test-back:
runs-on: ubuntu-latest
needs: build-mails
defaults:
run:
working-directory: src/backend
services:
postgres:
image: postgres:16
env:
POSTGRES_DB: people
POSTGRES_USER: dinum
POSTGRES_PASSWORD: pass
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
env:
DJANGO_CONFIGURATION: Test
DJANGO_SETTINGS_MODULE: people.settings
DJANGO_SECRET_KEY: ThisIsAnExampleKeyForTestPurposeOnly
OIDC_OP_JWKS_ENDPOINT: /endpoint-for-test-purpose-only
DB_HOST: localhost
DB_NAME: people
DB_USER: dinum
DB_PASSWORD: pass
DB_PORT: 5432
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Create writable /data
run: |
sudo mkdir -p /data/media && \
sudo mkdir -p /data/static
- name: Download mails' templates
uses: actions/download-artifact@v4
with:
name: mails-templates
path: src/backend/core/templates/mail
- name: Install Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install development dependencies
run: pip install --user .[dev]
- name: Install gettext (required to compile messages)
run: |
sudo apt-get update
sudo apt-get install -y gettext
- name: Generate a MO file from strings extracted from the project
run: python manage.py compilemessages
- name: Run tests
run: ~/.local/bin/pytest -n 2
i18n-crowdin:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "people,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
- name: Install gettext (required to make messages)
run: |
sudo apt-get update
sudo apt-get install -y gettext
- name: Install Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install development dependencies
working-directory: src/backend
run: pip install --user .[dev]
- name: Generate the translation base file
run: ~/.local/bin/django-admin makemessages --keep-pot --all
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/people/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '18.x'
cache: 'yarn'
cache-dependency-path: src/frontend/yarn.lock
- name: Install dependencies
run: cd src/frontend/ && yarn install --frozen-lockfile
- name: Download sources from Crowdin to stay synchronized
run: |
docker run \
--rm \
-e CROWDIN_API_TOKEN=$CROWDIN_API_TOKEN \
-e CROWDIN_PROJECT_ID=$CROWDIN_PROJECT_ID \
-e CROWDIN_BASE_PATH=$CROWDIN_BASE_PATH \
-v "${{ github.workspace }}:/app" \
crowdin/cli:3.16.0 \
crowdin download sources -c /app/crowdin/config.yml
- name: Extract the frontend translation
run: make frontend-i18n-extract
- name: Upload files to Crowdin
run: |
docker run \
--rm \
-e CROWDIN_API_TOKEN=$CROWDIN_API_TOKEN \
-e CROWDIN_PROJECT_ID=$CROWDIN_PROJECT_ID \
-e CROWDIN_BASE_PATH=$CROWDIN_BASE_PATH \
-v "${{ github.workspace }}:/app" \
crowdin/cli:3.16.0 \
crowdin upload sources -c /app/crowdin/config.yml

View File

@@ -1,34 +0,0 @@
name: Release Chart
run-name: Release Chart
on:
push:
paths:
- src/helm/impress/**
jobs:
release:
# depending on default permission settings for your org (contents being read-only or read-write for workloads), you will have to add permissions
# see: https://docs.github.com/en/actions/security-guides/automatic-token-authentication#modifying-the-permissions-for-the-github_token
permissions:
contents: write
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Cleanup
run: rm -rf ./src/helm/extra
- name: Install Helm
uses: azure/setup-helm@v4
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: Publish Helm charts
uses: numerique-gouv/helm-gh-pages@add-overwrite-option
with:
charts_dir: ./src/helm
token: ${{ secrets.GITHUB_TOKEN }}

8
.gitignore vendored
View File

@@ -30,10 +30,10 @@ MANIFEST
.next/
# Translations # Translations
*.mo
*.pot
# Environments
.env
.venv
env/
venv/
@@ -50,6 +50,9 @@ node_modules
# Mails
src/backend/core/templates/mail/
# Typescript client
src/frontend/tsclient
# Swagger
**/swagger.json
@@ -75,4 +78,5 @@ db.sqlite3
.idea/
.vscode/
*.iml
.devcontainer
.devcontainer/
.tool-versions

3
.gitmodules vendored
View File

@@ -0,0 +1,3 @@
[submodule "secrets"]
path = secrets
url = ../secrets

View File

@@ -1,10 +0,0 @@
creation_rules:
- path_regex: ./*
key_groups:
- age:
- age15fyxdwmg5mvldtqqus87xspuws2u0cpvwheehrtvkexj4tnsqqysw6re2x # jacques
- age16hnlml8yv4ynwy0seer57g8qww075crd0g7nsundz3pj4wk7m3vqftszg7 # github-repo
- age1plkp8td6zzfcavjusmsfrlk54t9vn8jjxm8zaz7cmnr7kzl2nfnsd54hwg # Anthony Le-Courric
- age12g6f5fse25tgrwweleh4jls3qs52hey2edh759smulwmk5lnzadslu2cp3 # Antoine Lebaud
- age1hnhuzj96ktkhpyygvmz0x9h8mfvssz7ss6emmukags644mdhf4msajk93r # Samuel Paccoud

View File

@@ -7,661 +7,3 @@ and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Added
- ✨(frontend) add customization for translations #857
- 📝(project) add troubleshoot doc #1066
- 📝(project) add system-requirement doc #1066
- 🔧(front) configure x-frame-options to DENY in nginx conf #1084
- (doc) add documentation to install with compose #855
- ✨(backend) allow to disable checking unsafe mimetype on attachment upload
- ✨Ask for access #1081
### Changed
- 📌(yjs) stop pinning node to minor version on yjs docker image #1005
- 🧑‍💻(docker) add .next to .dockerignore #1055
- 🧑‍💻(docker) handle frontend development images with docker compose #1033
- 🧑‍💻(docker) add y-provider config to development environment #1057
- ⚡️(frontend) optimize document fetch error handling #1089
### Fixed
- 🐛(frontend) table of content disappearing #982
- 🐛(frontend) fix multiple EmojiPicker #1012
- 🐛(frontend) fix meta title #1017
- 🔧(git) set LF line endings for all text files #1032
- 📝(docs) minor fixes to docs/env.md
- ✨(backend) support `_FILE` environment variables for secrets #912
- ✨(frontend) support `_FILE` environment variables for secrets #912
### Removed
- 🔥(frontend) remove Beta from logo #1095
## [3.3.0] - 2025-05-06
### Added
- ✨(backend) add endpoint checking media status #984
- ✨(backend) allow setting session cookie age via env var #977
- ✨(backend) allow theme customnization using a configuration file #948
- ✨(frontend) Add a custom callout block to the editor #892
- 🚩(frontend) version MIT only #911
- ✨(backend) integrate maleware_detection from django-lasuite #936
- 🏗️(frontend) Footer configurable #959
- 🩺(CI) add lint spell mistakes #954
- ✨(frontend) create generic theme #792
- 🛂(frontend) block edition to not connected users #945
- 🚸(frontend) Let loader during upload analyze #984
- 🚩(frontend) feature flag on blocking edition #997
### Changed
- 📝(frontend) Update documentation #949
- ✅(frontend) Improve tests coverage #949
- ⬆️(docker) upgrade backend image to python 3.13 #973
- ⬆️(docker) upgrade node images to alpine 3.21 #973
### Fixed
- 🐛(y-provider) increase JSON size limits for transcription conversion #989
### Removed
- 🔥(back) remove footer endpoint #948
## [3.2.1] - 2025-05-06
## Fixed
- 🐛(frontend) fix list copy paste #943
- 📝(doc) update contributing policy (commit signatures are now mandatory) #895
## [3.2.0] - 2025-05-05
## Added
- ✨(frontend) multi-pages #701
- ✨(backend) include ancestors accesses on document accesses list view #846
- ✨(backend) add ancestors links reach and role to document API #846
- 🚸(backend) make document search on title accent-insensitive #874
- 🚩 add homepage feature flag #861
- 📝(doc) update contributing policy (commit signatures are now mandatory) #895
- ✨(settings) Allow configuring PKCE for the SSO #886
- 🌐(i18n) activate chinese and spanish languages #884
- 🔧(backend) allow overwriting the data directory #893
- (backend) add `django-lasuite` dependency #839
- ✨(frontend) advanced table features #908
## Changed
- ♻️(backend) stop requiring owner for non-root documents #846
- ♻️(backend) simplify roles by ranking them and return only the max role #846
- ⚡️(frontend) reduce unblocking time for config #867
- ♻️(frontend) bind UI with ability access #900
- ♻️(frontend) use built-in Quote block #908
## Fixed
- 🐛(backend) fix link definition select options linked to ancestors #846
- 🐛(nginx) fix 404 when accessing a doc #866
- 🔒️(drf) disable browsable HTML API renderer #919
- 🔒(frontend) enhance file download security #889
- 🐛(backend) race condition create doc #633
- 🐛(frontend) fix breaklines in custom blocks #908
## Fixed
- 🐛(backend) fix link definition select options linked to ancestors #846
## [3.1.0] - 2025-04-07
## Added
- ✨(backend) add ancestors links definitions to document abilities #846
- 🚩(backend) add feature flag for the footer #841
- 🔧(backend) add view to manage footer json #841
- ✨(frontend) add custom css style #771
- 🚩(frontend) conditionally render AI button only when feature is enabled #814
## Changed
- 🚨(frontend) block button when creating doc #749
## Fixed
- 🐛(backend) fix link definition select options linked to ancestors #846
- 🐛(back) validate document content in serializer #822
- 🐛(frontend) fix selection click past end of content #840
## [3.0.0] - 2025-03-28
## Added
- 📄(legal) Require contributors to sign a DCO #779
## Changed
- ♻️(frontend) Integrate UI kit #783
- 🏗️(y-provider) manage auth in y-provider app #804
## Fixed
- 🐛(backend) compute ancestor_links in get_abilities if needed #725
- 🔒️(back) restrict access to document accesses #801
## [2.6.0] - 2025-03-21
## Added
- 📝(doc) add publiccode.yml #770
## Changed
- 🚸(frontend) ctrl+k modal not when editor is focused #712
## Fixed
- 🐛(back) allow only images to be used with the cors-proxy #781
- 🐛(backend) stop returning inactive users on the list endpoint #636
- 🔒️(backend) require at least 5 characters to search for users #636
- 🔒️(back) throttle user list endpoint #636
- 🔒️(back) remove pagination and limit to 5 for user list endpoint #636
## [2.5.0] - 2025-03-18
## Added
- 📝(doc) Added GNU Make link to README #750
- ✨(frontend) add pinning on doc detail #711
- 🚩(frontend) feature flag analytic on copy as html #649
- ✨(frontend) Custom block divider with export #698
- 🌐(i18n) activate dutch language #742
- ✨(frontend) add Beautify action to AI transform #478
- ✨(frontend) add Emojify action to AI transform #478
## Changed
- 🧑‍💻(frontend) change literal section open source #702
- ♻️(frontend) replace cors proxy for export #695
- 🚨(gitlint) Allow uppercase in commit messages #756
- ♻️(frontend) Improve AI translations #478
## Fixed
- 🐛(frontend) SVG export #706
- 🐛(frontend) remove scroll listener table content #688
- 🔒️(back) restrict access to favorite_list endpoint #690
- 🐛(backend) refactor to fix filtering on children
and descendants views #695
- 🐛(action) fix notify-argocd workflow #713
- 🚨(helm) fix helmfile lint #736
- 🚚(frontend) redirect to 401 page when 401 error #759
## [2.4.0] - 2025-03-06
## Added
- ✨(frontend) synchronize language-choice #401
## Changed
- Use sentry tags instead of extra scope
## Fixed
- 🐛(frontend) fix collaboration error #684
## [2.3.0] - 2025-03-03
## Added
- ✨(backend) limit link reach/role select options depending on ancestors #645
- ✨(backend) add new "descendants" action to document API endpoint #645
- ✨(backend) new "tree" action on document detail endpoint #645
- ✨(backend) allow forcing page size within limits #645
- 💄(frontend) add error pages #643
- 🔒️ Manage unsafe attachments #663
- ✨(frontend) Custom block quote with export #646
- ✨(frontend) add open source section homepage #666
- ✨(frontend) synchronize language-choice #401
## Changed
- 🛂(frontend) Restore version visibility #629
- 📝(doc) minor README.md formatting and wording enhancements
-Stop setting a default title on doc creation #634
- ♻️(frontend) misc ui improvements #644
## Fixed
- 🐛(backend) allow any type of extensions for media download #671
- ♻️(frontend) improve table pdf rendering
- 🐛(email) invitation emails in receivers language
## [2.2.0] - 2025-02-10
## Added
- 📝(doc) Add security.md and codeofconduct.md #604
- ✨(frontend) add home page #608
- ✨(frontend) cursor display on activity #609
- ✨(frontend) Add export page break #623
## Changed
- 🔧(backend) make AI feature reach configurable #628
## Fixed
- 🌐(CI) Fix email partially translated #616
- 🐛(frontend) fix cursor breakline #609
- 🐛(frontend) fix style pdf export #609
## [2.1.0] - 2025-01-29
## Added
- ✨(backend) add duplicate action to the document API endpoint
- ⚗️(backend) add util to extract text from base64 yjs document
- ✨(backend) add soft delete and restore API endpoints to documents #516
- ✨(backend) allow organizing documents in a tree structure #516
- ✨(backend) add "excerpt" field to document list serializer #516
- ✨(backend) add github actions to manage Crowdin workflow #559 & #563
- 📈Integrate Posthog #540
- 🏷️(backend) add content-type to uploaded files #552
- ✨(frontend) export pdf docx front side #537
## Changed
- 💄(frontend) add abilities on doc row #581
- 💄(frontend) improve DocsGridItem responsive padding #582
- 🔧(backend) Bump maximum page size to 200 #516
- 📝(doc) Improve Read me #558
## Fixed
- 🐛Fix invitations #575
## Removed
- 🔥(backend) remove "content" field from list serializer # 516
## [2.0.1] - 2025-01-17
## Fixed
-🐛(frontend) share modal is shown when you don't have the abilities #557
-🐛(frontend) title copy break app #564
## [2.0.0] - 2025-01-13
## Added
- 🔧(backend) add option to configure list of essential OIDC claims #525 & #531
- 🔧(helm) add option to disable default tls setting by @dominikkaminski #519
- 💄(frontend) Add left panel #420
- 💄(frontend) add filtering to left panel #475
- ✨(frontend) new share modal ui #489
- ✨(frontend) add favorite feature #515
- 📝(documentation) Documentation about self-hosted installation #530
- ✨(helm) helm versioning #530
## Changed
- 🏗️(yjs-server) organize yjs server #528
- ♻️(frontend) better separation collaboration process #528
- 💄(frontend) updating the header and leftpanel for responsive #421
- 💄(frontend) update DocsGrid component #431
- 💄(frontend) update DocsGridOptions component #432
- 💄(frontend) update DocHeader ui #448
- 💄(frontend) update doc versioning ui #463
- 💄(frontend) update doc summary ui #473
- 📝(doc) update readme.md to match V2 changes #558 & #572
## Fixed
- 🐛(backend) fix create document via s2s if sub unknown but email found #543
- 🐛(frontend) hide search and create doc button if not authenticated #555
- 🐛(backend) race condition creation issue #556
## [1.10.0] - 2024-12-17
## Added
- ✨(backend) add server-to-server API endpoint to create documents #467
- ✨(email) white brand email #412
- ✨(y-provider) create a markdown converter endpoint #488
## Changed
- ⚡️(docker) improve y-provider image #422
## Fixed
- ⚡️(e2e) reduce flakiness on e2e tests #511
## Fixed
- 🐛(frontend) update doc editor height #481
- 💄(frontend) add doc search #485
## [1.9.0] - 2024-12-11
## Added
- ✨(backend) annotate number of accesses on documents in list view #429
- ✨(backend) allow users to mark/unmark documents as favorite #429
## Changed
- 🔒️(collaboration) increase collaboration access security #472
- 🔨(frontend) encapsulated title to its own component #474
- ⚡️(backend) optimize number of queries on document list view #429
- ♻️(frontend) stop to use provider with version #480
- 🚚(collaboration) change the websocket key name #480
## Fixed
- 🐛(frontend) fix initial content with collaboration #484
- 🐛(frontend) Fix hidden menu on Firefox #468
- 🐛(backend) fix sanitize problem IA #490
## [1.8.2] - 2024-11-28
## Changed
- ♻️(SW) change strategy html caching #460
## [1.8.1] - 2024-11-27
## Fixed
- 🐛(frontend) link not clickable and flickering firefox #457
## [1.8.0] - 2024-11-25
## Added
- 🌐(backend) add German translation #259
- 🌐(frontend) add German translation #255
- ✨(frontend) add a broadcast store #387
- ✨(backend) whitelist pod's IP address #443
- ✨(backend) config endpoint #425
- ✨(frontend) config endpoint #424
- ✨(frontend) add sentry #424
- ✨(frontend) add crisp chatbot #450
## Changed
- 🚸(backend) improve users similarity search and sort results #391
- ♻️(frontend) simplify stores #402
- ✨(frontend) update $css Box props type to add styled components RuleSet #423
- ✅(CI) trivy continue on error #453
## Fixed
- 🔧(backend) fix logging for docker and make it configurable by envar #427
- 🦺(backend) add comma to sub regex #408
- 🐛(editor) collaborative user tag hidden when read only #385
- 🐛(frontend) users have view access when revoked #387
- 🐛(frontend) fix placeholder editable when double clicks #454
## [1.7.0] - 2024-10-24
## Added
- 📝Contributing.md #352
- 🌐(frontend) add localization to editor #368
- ✨Public and restricted doc editable #357
- ✨(frontend) Add full name if available #380
- ✨(backend) Add view accesses ability #376
## Changed
- ♻️(frontend) list accesses if user has abilities #376
- ♻️(frontend) avoid documents indexing in search engine #372
- 👔(backend) doc restricted by default #388
## Fixed
- 🐛(backend) require right to manage document accesses to see invitations #369
- 🐛(i18n) same frontend and backend language using shared cookies #365
- 🐛(frontend) add default toolbar buttons #355
- 🐛(frontend) throttle error correctly display #378
## Removed
- 🔥(helm) remove infra related codes #366
## [1.6.0] - 2024-10-17
## Added
- ✨AI to doc editor #250
- ✨(backend) allow uploading more types of attachments #309
- ✨(frontend) add buttons to copy document to clipboard as HTML/Markdown #318
## Changed
- ♻️(frontend) more multi theme friendly #325
- ♻️ Bootstrap frontend #257
- ♻️ Add username in email #314
## Fixed
- 🛂(backend) do not duplicate user when disabled
- 🐛(frontend) invalidate queries after removing user #336
- 🐛(backend) Fix dysfunctional permissions on document create #329
- 🐛(backend) fix nginx docker container #340
- 🐛(frontend) fix copy paste firefox #353
## [1.5.1] - 2024-10-10
## Fixed
- 🐛(db) fix users duplicate #316
## [1.5.0] - 2024-10-09
## Added
- ✨(backend) add name fields to the user synchronized with OIDC #301
- ✨(ci) add security scan #291
- ♻️(frontend) Add versions #277
- ✨(frontend) one-click document creation #275
- ✨(frontend) edit title inline #275
- 📱(frontend) mobile responsive #304
- 🌐(frontend) Update translation #308
## Changed
- 💄(frontend) error alert closeable on editor #284
- ♻️(backend) Change email content #283
- 🛂(frontend) viewers and editors can access share modal #302
- ♻️(frontend) remove footer on doc editor #313
## Fixed
- 🛂(frontend) match email if no existing user matches the sub
- 🐛(backend) gitlab oicd userinfo endpoint #232
- 🛂(frontend) redirect to the OIDC when private doc and unauthentified #292
- ♻️(backend) getting list of document versions available for a user #258
- 🔧(backend) fix configuration to avoid different ssl warning #297
- 🐛(frontend) fix editor break line not working #302
## [1.4.0] - 2024-09-17
## Added
- ✨Add link public/authenticated/restricted access with read/editor roles #234
- ✨(frontend) add copy link button #235
- 🛂(frontend) access public docs without being logged #235
## Changed
- ♻️(backend) Allow null titles on documents for easier creation #234
- 🛂(backend) stop to list public doc to everyone #234
- 🚚(frontend) change visibility in share modal #235
- ⚡️(frontend) Improve summary #244
## Fixed
- 🐛(backend) Fix forcing ID when creating a document via API endpoint #234
- 🐛 Rebuild frontend dev container from makefile #248
## [1.3.0] - 2024-09-05
## Added
- ✨Add image attachments with access control
- ✨(frontend) Upload image to a document #211
- ✨(frontend) Summary #223
- ✨(frontend) update meta title for docs page #231
## Changed
- 💄(frontend) code background darkened on editor #214
- 🔥(frontend) hide markdown button if not text #213
## Fixed
- 🐛 Fix emoticon in pdf export #225
- 🐛 Fix collaboration on document #226
- 🐛 (docker) Fix compatibility with mac #230
## Removed
- 🔥(frontend) remove saving modal #213
## [1.2.1] - 2024-08-23
## Changed
- ♻️ Change ordering docs datagrid #195
- 🔥(helm) use scaleway email #194
## [1.2.0] - 2024-08-22
## Added
- 🎨(frontend) better conversion editor to pdf #151
- ✨Export docx (word) #161
- 🌐Internationalize invitation email #167
- ✨(frontend) White branding #164
- ✨Email invitation when add user to doc #171
- ✨Invitation management #174
## Fixed
- 🐛(y-webrtc) fix prob connection #147
- ⚡️(frontend) improve select share stability #159
- 🐛(backend) enable SSL when sending email #165
## Changed
- 🎨(frontend) stop limit layout height to screen size #158
- ⚡️(CI) only e2e chrome mandatory #177
## Removed
- 🔥(helm) remove htaccess #181
## [1.1.0] - 2024-07-15
## Added
- 🤡(demo) generate dummy documents on dev users #120
- ✨(frontend) create side modal component #134
- ✨(frontend) Doc grid actions (update / delete) #136
- ✨(frontend) Doc editor header information #137
## Changed
- ♻️(frontend) replace docs panel with docs grid #120
- ♻️(frontend) create a doc from a modal #132
- ♻️(frontend) manage members from the share modal #140
## [1.0.0] - 2024-07-02
## Added
- 🛂(frontend) Manage the document's right (#75)
- ✨(frontend) Update document (#68)
- ✨(frontend) Remove document (#68)
- 🐳(docker) dockerize dev frontend (#63)
- 👔(backend) list users with email filtering (#79)
- ✨(frontend) add user to a document (#52)
- ✨(frontend) invite user to a document (#52)
- 🛂(frontend) manage members (update role / list / remove) (#81)
- ✨(frontend) offline mode (#88)
- 🌐(frontend) translate cgu (#83)
- ✨(service-worker) offline doc management (#94)
- ⚗️(frontend) Add beta tag on logo (#121)
## Changed
- ♻️(frontend) Change site from Impress to Docs (#76)
- ✨(frontend) Generate PDF from a modal (#68)
- 🔧(helm) sticky session by request_uri for signaling server (#78)
- ♻️(frontend) change logo (#84)
- ♻️(frontend) pdf has title doc (#84)
- ⚡️(e2e) unique login between tests (#80)
- ⚡️(CI) improve e2e job (#86)
- ♻️(frontend) improve the error and message info ui (#93)
- ✏️(frontend) change all occurrences of pad to doc (#99)
## Fixed
- 🐛(frontend) Fix the break line when generate PDF (#84)
## Delete
- 💚(CI) Remove trigger workflow on push tags on CI (#68)
- 🔥(frontend) Remove coming soon page (#121)
## [0.1.0] - 2024-05-24
## Added
- ✨(frontend) Coming Soon page (#67)
- 🚀 Impress, project to manage your documents easily and collaboratively.
[unreleased]: https://github.com/numerique-gouv/impress/compare/v3.3.0...main
[v3.3.0]: https://github.com/numerique-gouv/impress/releases/v3.3.0
[v3.2.1]: https://github.com/numerique-gouv/impress/releases/v3.2.1
[v3.2.0]: https://github.com/numerique-gouv/impress/releases/v3.2.0
[v3.1.0]: https://github.com/numerique-gouv/impress/releases/v3.1.0
[v3.0.0]: https://github.com/numerique-gouv/impress/releases/v3.0.0
[v2.6.0]: https://github.com/numerique-gouv/impress/releases/v2.6.0
[v2.5.0]: https://github.com/numerique-gouv/impress/releases/v2.5.0
[v2.4.0]: https://github.com/numerique-gouv/impress/releases/v2.4.0
[v2.3.0]: https://github.com/numerique-gouv/impress/releases/v2.3.0
[v2.2.0]: https://github.com/numerique-gouv/impress/releases/v2.2.0
[v2.1.0]: https://github.com/numerique-gouv/impress/releases/v2.1.0
[v2.0.1]: https://github.com/numerique-gouv/impress/releases/v2.0.1
[v2.0.0]: https://github.com/numerique-gouv/impress/releases/v2.0.0
[v1.10.0]: https://github.com/numerique-gouv/impress/releases/v1.10.0
[v1.9.0]: https://github.com/numerique-gouv/impress/releases/v1.9.0
[v1.8.2]: https://github.com/numerique-gouv/impress/releases/v1.8.2
[v1.8.1]: https://github.com/numerique-gouv/impress/releases/v1.8.1
[v1.8.0]: https://github.com/numerique-gouv/impress/releases/v1.8.0
[v1.7.0]: https://github.com/numerique-gouv/impress/releases/v1.7.0
[v1.6.0]: https://github.com/numerique-gouv/impress/releases/v1.6.0
[1.5.1]: https://github.com/numerique-gouv/impress/releases/v1.5.1
[1.5.0]: https://github.com/numerique-gouv/impress/releases/v1.5.0
[1.4.0]: https://github.com/numerique-gouv/impress/releases/v1.4.0
[1.3.0]: https://github.com/numerique-gouv/impress/releases/v1.3.0
[1.2.1]: https://github.com/numerique-gouv/impress/releases/v1.2.1
[1.2.0]: https://github.com/numerique-gouv/impress/releases/v1.2.0
[1.1.0]: https://github.com/numerique-gouv/impress/releases/v1.1.0
[1.0.0]: https://github.com/numerique-gouv/impress/releases/v1.0.0
[0.1.0]: https://github.com/numerique-gouv/impress/releases/v0.1.0

View File

@@ -1,79 +0,0 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our community include:
- Demonstrating empathy and kindness toward other people
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
- Focusing on what is best not just for us as individuals, but for the overall community
Examples of unacceptable behavior include:
- The use of sexualized language or imagery, and sexual attention or advances of any kind
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or email address, without their explicit permission
- Other conduct which could reasonably be considered inappropriate in a professional setting
## Enforcement Responsibilities
- Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.
- Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.
## Scope
- This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
## Enforcement
- Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at docs@numerique.gouv.fr.
- All complaints will be reviewed and investigated promptly and fairly.
- All community leaders are obligated to respect the privacy and security of the reporter of any incident.
## Enforcement Guidelines
- Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of the following Code of Conduct
## Code of Conduct:
### 1. Correction
Community Impact: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
Consequence: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.
### 2. Warning
Community Impact: A violation through a single incident or series of actions.
Consequence: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.
### 3. Temporary Ban
Community Impact: A serious violation of community standards, including sustained inappropriate behavior.
Consequence: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
Community Impact: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
Consequence: A permanent ban from any sort of public interaction within the community.
## Attribution
This Code of Conduct is adapted from the Contributor Covenant, version 2.1, available at https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.
Community Impact Guidelines were inspired by Mozilla's [code of conduct enforcement ladder](https://github.com/mozilla/inclusion/blob/master/code-of-conduct-enforcement/consequence-ladder.md).
For answers to common questions about this code of conduct, see the FAQ at https://www.contributor-covenant.org/faq. Translations are available at https://www.contributor-covenant.org/translations.

View File

@@ -1,102 +0,0 @@
# Contributing to the Project
Thank you for taking the time to contribute! Please follow these guidelines to ensure a smooth and productive workflow. 🚀🚀🚀
To get started with the project, please refer to the [README.md](https://github.com/suitenumerique/docs/blob/main/README.md) for detailed instructions on how to run Docs locally.
Contributors are required to sign off their commits with `git commit --signoff`: this confirms that they have read and accepted the [Developer's Certificate of Origin 1.1](https://developercertificate.org/). For security reasons we also require [signing your commits with your SSH or GPG key](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification) with `git commit -S`.
Please also check out our [dev handbook](https://suitenumerique.gitbook.io/handbook) to learn our best practices.
## Help us with translations
You can help us with translations on [Crowdin](https://crowdin.com/project/lasuite-docs).
Your language is not there? Request it on our Crowdin page 😊 or ping us on [Matrix](https://matrix.to/#/#docs-official:matrix.org) and let us know if you can help with translations and/or proofreading.
## Creating an Issue
When creating an issue, please provide the following details:
1. **Title**: A concise and descriptive title for the issue.
2. **Description**: A detailed explanation of the issue, including relevant context or screenshots if applicable.
3. **Steps to Reproduce**: If the issue is a bug, include the steps needed to reproduce the problem.
4. **Expected vs. Actual Behavior**: Describe what you expected to happen and what actually happened.
5. **Labels**: Add appropriate labels to categorize the issue (e.g., bug, feature request, documentation).
## Selecting an issue
We use a [GitHub Project](https://github.com/orgs/numerique-gouv/projects/13) in order to prioritize our workload.
Please check in priority the issues that are in the **todo** column and have a higher priority (P0 -> P2).
## Commit Message Format
All commit messages must adhere to the following format:
`<gitmoji>(type) title description`
* <**gitmoji**>: Use a gitmoji to represent the purpose of the commit. For example, ✨ for adding a new feature or 🔥 for removing something, see the list [here](https://gitmoji.dev/).
* **(type)**: Describe the type of change. Common types include `backend`, `frontend`, `CI`, `docker` etc...
* **title**: A short, descriptive title for the change (*)
* **blank line after the commit title
* **description**: Include additional details on why you made the changes (**).
(*) ⚠️ **Make sure you add no space between the emoji and the (type) but add a space after the closing parenthesis of the type and use no caps!**
(**) ⚠️ **Commit description message is mandatory and shouldn't be too long**
### Example Commit Message
```
✨(frontend) add user authentication logic
Implemented login and signup features, and integrated OAuth2 for social login.
```
## Changelog Update
Please add a line to the changelog describing your development. The changelog entry should include a brief summary of the changes, this helps in tracking changes effectively and keeping everyone informed. We usually include the title of the pull request, followed by the pull request ID to finish the log entry. The changelog line should be less than 80 characters in total.
### Example Changelog Message
```
## [Unreleased]
## Added
- ✨(frontend) add AI to the project #321
```
## Pull Requests
It is nice to add information about the purpose of the pull request to help reviewers understand the context and intent of the changes. If you can, add some pictures or a small video to show the changes.
### Don't forget to:
- signoff your commits
- sign your commits with your key (SSH, GPG etc.)
- check your commits (see warnings above)
- check the linting: `make lint && make frontend-lint`
- check the tests: `make test`
- add a changelog entry
Once all the required tests have passed, you can request a review from the project maintainers.
## Code Style
Please maintain consistency in code style. Run any linting tools available to make sure the code is clean and follows the project's conventions.
## Tests
Make sure that all new features or fixes have corresponding tests. Run the test suite before pushing your changes to ensure that nothing is broken.
## Asking for Help
If you need any help while contributing, feel free to open a discussion or ask for guidance in the issue tracker. We are more than happy to assist!
Thank you for your contributions! 👍
## Contribute to BlockNote
We use [BlockNote](https://www.blocknotejs.org/) for the text editing features of Docs.
If you find and issue with the editor you can [report it](https://github.com/TypeCellOS/BlockNote/issues) directly on their repository.
Please consider contributing to BlockNotejs, as a library, it's useful to many projects not just Docs.
The project is licended with Mozilla Public License Version 2.0 but be aware that [XL packages](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-docx-exporter/LICENSE) are dual licenced with GNU AFFERO GENERAL PUBLIC LICENCE Version 3 and proprietary licence if you are [sponsor](https://www.blocknotejs.org/pricing).

View File

@@ -1,26 +1,69 @@
# Django impress
# Django People
# ---- base image to inherit from ----
FROM python:3.13.3-alpine AS base
FROM python:3.10-slim-bullseye as base
# Upgrade pip to its latest release to speed up dependencies installation
RUN python -m pip install --upgrade pip setuptools
RUN python -m pip install --upgrade pip
# Upgrade system packages to install security updates
RUN apk update && \
apk upgrade
RUN apt-get update && \
apt-get -y upgrade && \
rm -rf /var/lib/apt/lists/*
# ---- Back-end builder image ----
FROM base AS back-builder
### ---- Front-end dependencies image ----
FROM node:20 as frontend-deps
WORKDIR /deps
COPY ./src/frontend/package.json ./package.json
COPY ./src/frontend/yarn.lock ./yarn.lock
COPY ./src/frontend/apps/desk/package.json ./apps/desk/package.json
COPY ./src/frontend/packages/i18n/package.json ./packages/i18n/package.json
COPY ./src/frontend/packages/eslint-config-people/package.json ./packages/eslint-config-people/package.json
RUN yarn --frozen-lockfile
### ---- Front-end builder dev image ----
FROM node:20 as frontend-builder-dev
WORKDIR /builder
# Install Rust and Cargo using Alpine's package manager
RUN apk add --no-cache \
build-base \
libffi-dev \
rust \
cargo
COPY --from=frontend-deps /deps/node_modules ./node_modules
COPY ./src/frontend .
WORKDIR ./apps/desk
### ---- Front-end builder image ----
FROM frontend-builder-dev as frontend-builder
RUN yarn build
# ---- Front-end image ----
FROM nginxinc/nginx-unprivileged:1.25 as frontend-production
# Un-privileged user running the application
ARG DOCKER_USER
USER ${DOCKER_USER}
COPY --from=frontend-builder \
/builder/apps/desk/out \
/usr/share/nginx/html
COPY ./src/frontend/apps/desk/conf/default.conf /etc/nginx/conf.d
# Copy entrypoint
COPY ./docker/files/usr/local/bin/entrypoint /usr/local/bin/entrypoint
ENTRYPOINT [ "/usr/local/bin/entrypoint" ]
CMD ["nginx", "-g", "daemon off;"]
# ---- Back-end builder image ----
FROM base as back-builder
WORKDIR /builder
# Copy required python dependencies
COPY ./src/backend /builder
@@ -30,7 +73,7 @@ RUN mkdir /install && \
# ---- mails ----
FROM node:24 AS mail-builder
FROM node:20 as mail-builder
COPY ./src/mail /mail/app
@@ -41,48 +84,48 @@ RUN yarn install --frozen-lockfile && \
# ---- static link collector ----
FROM base AS link-collector
ARG IMPRESS_STATIC_ROOT=/data/static
FROM base as link-collector
ARG PEOPLE_STATIC_ROOT=/data/static
# Install pango & rdfind
RUN apk add \
pango \
rdfind
# Install libpangocairo & rdfind
RUN apt-get update && \
apt-get install -y \
libpangocairo-1.0-0 \
rdfind && \
rm -rf /var/lib/apt/lists/*
# Copy installed python dependencies
COPY --from=back-builder /install /usr/local
# Copy impress application (see .dockerignore)
# Copy people application (see .dockerignore)
COPY ./src/backend /app/
WORKDIR /app
# collectstatic
RUN DJANGO_CONFIGURATION=Build \
RUN DJANGO_CONFIGURATION=Build DJANGO_JWT_PRIVATE_SIGNING_KEY=Dummy \
python manage.py collectstatic --noinput
# Replace duplicated file by a symlink to decrease the overall size of the
# final image
RUN rdfind -makesymlinks true -followsymlinks true -makeresultsfile false ${IMPRESS_STATIC_ROOT}
RUN rdfind -makesymlinks true -followsymlinks true -makeresultsfile false ${PEOPLE_STATIC_ROOT}
# ---- Core application image ----
FROM base AS core
FROM base as core
ENV PYTHONUNBUFFERED=1
# Install required system libs
RUN apk add \
cairo \
file \
font-noto \
font-noto-emoji \
gettext \
gdk-pixbuf \
libffi-dev \
pango \
shared-mime-info
RUN wget https://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/conf/mime.types -O /etc/mime.types
RUN apt-get update && \
apt-get install -y \
gettext \
libcairo2 \
libffi-dev \
libgdk-pixbuf2.0-0 \
libpango-1.0-0 \
libpangocairo-1.0-0 \
shared-mime-info && \
rm -rf /var/lib/apt/lists/*
# Copy entrypoint
COPY ./docker/files/usr/local/bin/entrypoint /usr/local/bin/entrypoint
@@ -95,33 +138,30 @@ RUN chmod g=u /etc/passwd
# Copy installed python dependencies
COPY --from=back-builder /install /usr/local
# Copy impress application (see .dockerignore)
# Copy people application (see .dockerignore)
COPY ./src/backend /app/
WORKDIR /app
# Generate compiled translation messages
RUN DJANGO_CONFIGURATION=Build \
python manage.py compilemessages
# We wrap commands run in this container by the following entrypoint that
# creates a user on-the-fly with the container user ID (see USER) and root group
# ID.
ENTRYPOINT [ "/usr/local/bin/entrypoint" ]
# ---- Development image ----
FROM core AS backend-development
FROM core as backend-development
# Switch back to the root user to install development dependencies
USER root:root
# Install psql
RUN apk add postgresql-client
RUN apt-get update && \
apt-get install -y postgresql-client && \
rm -rf /var/lib/apt/lists/*
# Uninstall impress and re-install it in editable mode along with development
# Uninstall people and re-install it in editable mode along with development
# dependencies
RUN pip uninstall -y impress
RUN pip uninstall -y people
RUN pip install -e .[dev]
# Restore the un-privileged user running the application
@@ -137,26 +177,23 @@ ENV DB_HOST=postgresql \
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
# ---- Production image ----
FROM core AS backend-production
FROM core as backend-production
# Remove apk cache, we don't need it anymore
RUN rm -rf /var/cache/apk/*
ARG IMPRESS_STATIC_ROOT=/data/static
ARG PEOPLE_STATIC_ROOT=/data/static
# Gunicorn
RUN mkdir -p /usr/local/etc/gunicorn
COPY docker/files/usr/local/etc/gunicorn/impress.py /usr/local/etc/gunicorn/impress.py
COPY docker/files/usr/local/etc/gunicorn/people.py /usr/local/etc/gunicorn/people.py
# Un-privileged user running the application
ARG DOCKER_USER
USER ${DOCKER_USER}
# Copy statics
COPY --from=link-collector ${IMPRESS_STATIC_ROOT} ${IMPRESS_STATIC_ROOT}
COPY --from=link-collector ${PEOPLE_STATIC_ROOT} ${PEOPLE_STATIC_ROOT}
# Copy impress mails
# Copy people mails
COPY --from=mail-builder /mail/backend/core/templates/mail /app/core/templates/mail
# The default command runs gunicorn WSGI server in impress's main module
CMD ["gunicorn", "-c", "/usr/local/etc/gunicorn/impress.py", "impress.wsgi:application"]
# The default command runs gunicorn WSGI server in people's main module
CMD ["gunicorn", "-c", "/usr/local/etc/gunicorn/people.py", "people.wsgi:application"]

171
Makefile
View File

@@ -39,20 +39,22 @@ DOCKER_UID = $(shell id -u)
DOCKER_GID = $(shell id -g)
DOCKER_USER = $(DOCKER_UID):$(DOCKER_GID)
COMPOSE = DOCKER_USER=$(DOCKER_USER) docker compose
COMPOSE_E2E = DOCKER_USER=$(DOCKER_USER) docker compose -f compose.yml -f compose-e2e.yml
COMPOSE_EXEC = $(COMPOSE) exec
COMPOSE_EXEC_APP = $(COMPOSE_EXEC) app-dev
COMPOSE_RUN = $(COMPOSE) run --rm
COMPOSE_RUN_APP = $(COMPOSE_RUN) app-dev
COMPOSE_RUN_CROWDIN = $(COMPOSE_RUN) crowdin crowdin
WAIT_DB = @$(COMPOSE_RUN) dockerize -wait tcp://$(DB_HOST):$(DB_PORT) -timeout 60s
WAIT_KC_DB = $(COMPOSE_RUN) dockerize -wait tcp://kc_postgresql:5432 -timeout 60s
# -- Backend
MANAGE = $(COMPOSE_RUN_APP) python manage.py
MAIL_YARN = $(COMPOSE_RUN) -w /app/src/mail node yarn
TSCLIENT_YARN = $(COMPOSE_RUN) -w /app/src/tsclient node yarn
# -- Frontend
PATH_FRONT = ./src/frontend
PATH_FRONT_IMPRESS = $(PATH_FRONT)/apps/impress
PATH_FRONT_DESK = $(PATH_FRONT)/apps/desk
# ==============================================================================
# RULES
@@ -75,101 +77,49 @@ create-env-files: \
env.d/development/kc_postgresql
.PHONY: create-env-files
pre-bootstrap: \
bootstrap: ## Prepare Docker images for the project and install frontend dependencies
bootstrap: \
data/media \
data/static \
create-env-files
.PHONY: pre-bootstrap
post-bootstrap: \
create-env-files \
build \
run \
migrate \
demo \
back-i18n-compile \
mails-install \
mails-build
.PHONY: post-bootstrap
bootstrap: ## Prepare Docker developmentimages for the project
bootstrap: \
pre-bootstrap \
build \
post-bootstrap \
run
mails-build \
install-front-desk
.PHONY: bootstrap
bootstrap-e2e: ## Prepare Docker production images to be used for e2e tests
bootstrap-e2e: \
pre-bootstrap \
build-e2e \
post-bootstrap \
run-e2e
.PHONY: bootstrap-e2e
# -- Docker/compose
build: cache ?=
build: ## build the project containers
@$(MAKE) build-backend cache=$(cache)
@$(MAKE) build-yjs-provider cache=$(cache)
@$(MAKE) build-frontend cache=$(cache)
build: ## build the app-dev container
@$(COMPOSE) build app-dev
.PHONY: build
build-backend: cache ?=
build-backend: ## build the app-dev container
@$(COMPOSE) build app-dev $(cache)
.PHONY: build-backend
build-yjs-provider: cache ?=
build-yjs-provider: ## build the y-provider container
@$(COMPOSE) build y-provider-development $(cache)
.PHONY: build-yjs-provider
build-frontend: cache ?=
build-frontend: ## build the frontend container
@$(COMPOSE) build frontend-development $(cache)
.PHONY: build-frontend
build-e2e: cache ?=
build-e2e: ## build the e2e container
@$(MAKE) build-backend cache=$(cache)
@$(COMPOSE_E2E) build frontend $(cache)
@$(COMPOSE_E2E) build y-provider $(cache)
.PHONY: build-e2e
down: ## stop and remove containers, networks, images, and volumes
@$(COMPOSE_E2E) down
@$(COMPOSE) down
.PHONY: down
logs: ## display app-dev logs (follow mode)
@$(COMPOSE) logs -f app-dev
.PHONY: logs
run-backend: ## Start only the backend application and all needed services
@$(COMPOSE) up --force-recreate -d celery-dev
@$(COMPOSE) up --force-recreate -d y-provider-development
@$(COMPOSE) up --force-recreate -d nginx
.PHONY: run-backend
run: ## start the wsgi (production) and development server
run:
@$(MAKE) run-backend
@$(COMPOSE) up --force-recreate -d frontend-development
@$(COMPOSE) up --force-recreate -d nginx
@$(COMPOSE) up --force-recreate -d app-dev
@$(COMPOSE) up --force-recreate -d celery-dev
@$(COMPOSE) up --force-recreate -d keycloak
@echo "Wait for postgresql to be up..."
@$(WAIT_KC_DB)
@$(WAIT_DB)
.PHONY: run
run-e2e: ## start the e2e server
run-e2e:
@$(MAKE) run-backend
@$(COMPOSE_E2E) stop y-provider-development
@$(COMPOSE_E2E) up --force-recreate -d frontend
@$(COMPOSE_E2E) up --force-recreate -d y-provider
.PHONY: run-e2e
status: ## an alias for "docker compose ps"
@$(COMPOSE_E2E) ps
@$(COMPOSE) ps
.PHONY: status
stop: ## stop the development server using Docker
@$(COMPOSE_E2E) stop
@$(COMPOSE) stop
.PHONY: stop
# -- Backend
@@ -216,21 +166,30 @@ test-back-parallel: ## run all back-end tests in parallel
bin/pytest -n auto $${args:-${1}}
.PHONY: test-back-parallel
makemigrations: ## run django makemigrations for the impress project.
makemigrations: ## run django makemigrations for the people project.
@echo "$(BOLD)Running makemigrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(MANAGE) makemigrations
@$(WAIT_DB)
@$(MANAGE) makemigrations $(ARGS)
.PHONY: makemigrations
migrate: ## run django migrations for the impress project.
migrate: ## run django migrations for the people project.
@echo "$(BOLD)Running migrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(MANAGE) migrate
@$(WAIT_DB)
@$(MANAGE) migrate $(ARGS)
.PHONY: migrate
showmigrations: ## run django showmigrations for the people project.
@echo "$(BOLD)Running showmigrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(WAIT_DB)
@$(MANAGE) showmigrations $(ARGS)
.PHONY: showmigrations
superuser: ## Create an admin superuser with password "admin"
@echo "$(BOLD)Creating a Django superuser$(RESET)"
@$(MANAGE) createsuperuser --email admin@example.com --password admin
@$(MANAGE) createsuperuser --admin_email admin@example.com --password admin
.PHONY: superuser
back-i18n-compile: ## compile the gettext files
@@ -238,7 +197,7 @@ back-i18n-compile: ## compile the gettext files
.PHONY: back-i18n-compile
back-i18n-generate: ## create the .pot files used for i18n
@$(MANAGE) makemessages -a --keep-pot --all
@$(MANAGE) makemessages -a --keep-pot
.PHONY: back-i18n-generate
shell: ## connect to database shell
@@ -272,7 +231,7 @@ env.d/development/kc_postgresql:
env.d/development/crowdin:
cp -n env.d/development/crowdin.dist env.d/development/crowdin
crowdin-download: ## Download translated message from crowdin
crowdin-download: ## Download translated message from Crowdin
@$(COMPOSE_RUN_CROWDIN) download -c crowdin/config.yml
.PHONY: crowdin-download
@@ -280,7 +239,7 @@ crowdin-download-sources: ## Download sources from Crowdin
@$(COMPOSE_RUN_CROWDIN) download sources -c crowdin/config.yml
.PHONY: crowdin-download-sources
crowdin-upload: ## Upload source translations to crowdin
crowdin-upload: ## Upload source translations to Crowdin
@$(COMPOSE_RUN_CROWDIN) upload sources -c crowdin/config.yml
.PHONY: crowdin-upload
@@ -327,30 +286,35 @@ mails-install: ## install the mail generator
@$(MAIL_YARN) install
.PHONY: mails-install
# -- TS client generator
tsclient-install: ## Install the Typescript API client generator
@$(TSCLIENT_YARN) install
.PHONY: tsclient-install
tsclient: tsclient-install ## Generate a Typescript API client
@$(TSCLIENT_YARN) generate:api:client:local ../frontend/tsclient
.PHONY: tsclient-install
# -- Misc
clean: ## restore repository state as it was freshly cloned
git clean -idx
.PHONY: clean
help:
@echo "$(BOLD)impress Makefile"
@echo "$(BOLD)People Makefile"
@echo "Please use 'make $(BOLD)target$(RESET)' where $(BOLD)target$(RESET) is one of:"
@grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(firstword $(MAKEFILE_LIST)) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "$(GREEN)%-30s$(RESET) %s\n", $$1, $$2}'
.PHONY: help
# Front
frontend-development-install: ## install the frontend locally
cd $(PATH_FRONT_IMPRESS) && yarn
.PHONY: frontend-development-install
# Front
install-front-desk: ## Install the frontend dependencies of app Desk
cd $(PATH_FRONT_DESK) && yarn
.PHONY: install-front-desk
frontend-lint: ## run the frontend linter
cd $(PATH_FRONT) && yarn lint
.PHONY: frontend-lint
run-frontend-development: ## Run the frontend in development mode
@$(COMPOSE) stop frontend-development
cd $(PATH_FRONT_IMPRESS) && yarn dev
.PHONY: run-frontend-development
run-front-desk: ## Start app Desk
cd $(PATH_FRONT_DESK) && yarn dev
.PHONY: run-front-desk
frontend-i18n-extract: ## Extract the frontend translation inside a json to be used for crowdin
cd $(PATH_FRONT) && yarn i18n:extract
@@ -367,21 +331,10 @@ frontend-i18n-compile: ## Format the crowin json files used deploy to the apps
.PHONY: frontend-i18n-compile
# -- K8S
build-k8s-cluster: ## build the kubernetes cluster using kind
start-kind: ## Create the kubernetes cluster
./bin/start-kind.sh
.PHONY: build-k8s-cluster
.PHONY: start-kind
start-tilt: ## start the kubernetes cluster using kind
tilt-up: ## start tilt - k8s local development
tilt up -f ./bin/Tiltfile
.PHONY: build-k8s-cluster
bump-packages-version: VERSION_TYPE ?= minor
bump-packages-version: ## bump the version of the project - VERSION_TYPE can be "major", "minor", "patch"
cd ./src/mail && yarn version --no-git-tag-version --$(VERSION_TYPE)
cd ./src/frontend/ && yarn version --no-git-tag-version --$(VERSION_TYPE)
cd ./src/frontend/apps/e2e/ && yarn version --no-git-tag-version --$(VERSION_TYPE)
cd ./src/frontend/apps/impress/ && yarn version --no-git-tag-version --$(VERSION_TYPE)
cd ./src/frontend/servers/y-provider/ && yarn version --no-git-tag-version --$(VERSION_TYPE)
cd ./src/frontend/packages/eslint-config-impress/ && yarn version --no-git-tag-version --$(VERSION_TYPE)
cd ./src/frontend/packages/i18n/ && yarn version --no-git-tag-version --$(VERSION_TYPE)
.PHONY: bump-packages-version
.PHONY: tilt-up

215
README.md
View File

@@ -1,213 +1,98 @@
<p align="center">
<a href="https://github.com/suitenumerique/docs">
<img alt="Docs" src="/docs/assets/banner-docs.png" width="100%" />
</a>
</p>
<p align="center">
<a href="https://github.com/suitenumerique/docs/stargazers/">
<img src="https://img.shields.io/github/stars/suitenumerique/docs" alt="">
</a>
<a href='https://github.com/suitenumerique/docs/blob/main/CONTRIBUTING.md'><img alt='PRs Welcome' src='https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=shields'/></a>
<img alt="GitHub commit activity" src="https://img.shields.io/github/commit-activity/m/suitenumerique/docs"/>
<img alt="GitHub closed issues" src="https://img.shields.io/github/issues-closed/suitenumerique/docs"/>
<a href="https://github.com/suitenumerique/docs/blob/main/LICENSE">
<img alt="MIT License" src="https://img.shields.io/github/license/suitenumerique/docs"/>
</a>
</p>
<p align="center">
<a href="https://matrix.to/#/#docs-official:matrix.org">
Chat on Matrix
</a> - <a href="/docs/">
Documentation
</a> - <a href="#getting-started-">
Getting started
</a> - <a href="mailto:docs@numerique.gouv.fr">
Reach out
</a>
</p>
# People
# La Suite Docs : Collaborative Text Editing
Docs, where your notes can become knowledge through live collaboration.
People is an application to handle users and teams.
<img src="/docs/assets/docs_live_collaboration_light.gif" width="100%" align="center"/>
As of today, this project is **not yet ready for production**. Expect breaking changes.
## Why use Docs ❓
Docs is a collaborative text editor designed to address common challenges in knowledge building and sharing.
People is built on top of [Django Rest
Framework](https://www.django-rest-framework.org/).
It offers a scalable and secure alternative to tools such as Google Docs, Notion (without the dbs), Outline, or Confluence.
## Getting started
### Write
* 😌 Get simple, accessible online editing for your team.
* 💅 Create clean documents with beautiful formatting options.
* 🖌️ Focus on your content using either the in-line editor, or [the Markdown syntax](https://www.markdownguide.org/basic-syntax/).
* 🧱 Quickly design your page thanks to the many block types, accessible from the `/` slash commands, as well as keyboard shortcuts.
* 🔌 Write offline! Your edits will be synced once you're back online.
* ✨ Save time thanks to our AI actions, such as rephrasing, summarizing, fixing typos, translating, etc. You can even turn your selected text into a prompt!
### Prerequisite
### Work together
* 🤝 Enjoy live editing! See your team collaborate in real time.
* 🔒 Keep your information secure thanks to granular access control. Only share with the right people.
* 📑 Export your content in multiple formats (`.odt`, `.docx`, `.pdf`) with customizable templates.
* 📚 Turn your team's collaborative work into organized knowledge with Subpages.
Make sure you have a recent version of Docker and [Docker
Compose](https://docs.docker.com/compose/install) installed on your laptop:
### Self-host
🚀 Docs is easy to install on your own servers
Available methods: Helm chart, Nix package
In the works: Docker Compose, YunoHost
⚠️ For some advanced features (ex: Export as PDF) Docs relies on XL packages from BlockNote. These are licenced under AGPL-3.0 and are not MIT compatible. You can perfectly use Docs without these packages by setting the environment variable `PUBLISH_AS_MIT` to true. That way you'll build an image of the application without the features that are not MIT compatible. Read the [environment variables documentation](/docs/env.md) for more information.
## Getting started 🔧
### Test it
You can test Docs on your browser by visiting this [demo document](https://impress-preprod.beta.numerique.gouv.fr/docs/6ee5aac4-4fb9-457d-95bf-bb56c2467713/)
### Run Docs locally
> ⚠️ The methods described below for running Docs locally is **for testing purposes only**. It is based on building Docs using [Minio](https://min.io/) as an S3-compatible storage solution. Of course you can choose any S3-compatible storage solution.
**Prerequisite**
Make sure you have a recent version of Docker and [Docker Compose](https://docs.docker.com/compose/install) installed on your laptop, then type:
```shellscript
```bash
$ docker -v
Docker version 20.10.2, build 2291f61
Docker version 20.10.2, build 2291f61
$ docker compose version
Docker Compose version v2.32.4
$ docker compose -v
docker compose version 1.27.4, build 40524192
```
> ⚠️ You may need to run the following commands with `sudo`, but this can be avoided by adding your user to the local `docker` group.
> ⚠️ You may need to run the following commands with `sudo` but this can be
> avoided by assigning your user to the `docker` group.
**Project bootstrap**
### Project bootstrap
The easiest way to start working on the project is to use [GNU Make](https://www.gnu.org/software/make/):
The easiest way to start working on the project is to use GNU Make:
```shellscript
$ make bootstrap FLUSH_ARGS='--no-input'
```bash
$ make bootstrap
```
This command builds the `app-dev` and `frontend-dev` containers, installs dependencies, performs database migrations and compiles translations. It's a good idea to use this command each time you are pulling code from the project repository to avoid dependency-related or migration-related issues.
This command builds the `app` container, installs dependencies, performs
database migrations and compile translations. It's a good idea to use this
command each time you are pulling code from the project repository to avoid
dependency-related or migration-related issues.
Your Docker services should now be up and running 🎉
You can access the project by going to <http://localhost:3000>.
Note that if you need to run them afterward, you can use the eponym Make rule:
You will be prompted to log in. The default credentials are:
```
username: impress
password: impress
```
📝 Note that if you need to run them afterwards, you can use the eponymous Make rule:
```shellscript
```bash
$ make run
```
⚠️ For the frontend developer, it is often better to run the frontend in development mode locally.
### Adding content
To do so, install the frontend dependencies with the following command:
You can create a basic demo site by running:
```shellscript
$ make frontend-development-install
```
$ make demo
And run the frontend locally in development mode with the following command:
Finally, you can check all available Make rules using:
```shellscript
$ make run-frontend-development
```
To start all the services, except the frontend container, you can use the following command:
```shellscript
$ make run-backend
```
**Adding content**
You can create a basic demo site by running this command:
```shellscript
$ make demo
```
Finally, you can check all available Make rules using this command:
```shellscript
```bash
$ make help
```
**Django admin**
### Django admin
You can access the Django admin site at:
<http://localhost:8071/admin>.
You can access the Django admin site at
[http://localhost:8071/admin](http://localhost:8071/admin).
You first need to create a superuser account:
```shellscript
```bash
$ make superuser
```
## Feedback 🙋‍♂️🙋‍♀️
You can then login with credentials `admin@example` / `admin`.
We'd love to hear your thoughts, and hear about your experiments, so come and say hi on [Matrix](https://matrix.to/#/#docs-official:matrix.org).
### Run frontend
## Roadmap 💡
Run the front with:
Want to know where the project is headed? [🗺️ Checkout our roadmap](https://github.com/orgs/numerique-gouv/projects/13/views/11)
## License 📝
This work is released under the MIT License (see [LICENSE](https://github.com/suitenumerique/docs/blob/main/LICENSE)).
While Docs is a public-driven initiative, our license choice is an invitation for private sector actors to use, sell and contribute to the project.
## Contributing 🙌
This project is intended to be community-driven, so please, do not hesitate to [get in touch](https://matrix.to/#/#docs-official:matrix.org) if you have any question related to our implementation or design decisions.
You can help us with translations on [Crowdin](https://crowdin.com/project/lasuite-docs).
If you intend to make pull requests, see [CONTRIBUTING](https://github.com/suitenumerique/docs/blob/main/CONTRIBUTING.md) for guidelines.
## Directory structure:
```markdown
docs
├── bin - executable scripts or binaries that are used for various tasks, such as setup scripts, utility scripts, or custom commands.
├── crowdin - for crowdin translations, a tool or service that helps manage translations for the project.
├── docker - Dockerfiles and related configuration files used to build Docker images for the project. These images can be used for development, testing, or production environments.
├── docs - documentation for the project, including user guides, API documentation, and other helpful resources.
├── env.d/development - environment-specific configuration files for the development environment. These files might include environment variables, configuration settings, or other setup files needed for development.
├── gitlint - configuration files for `gitlint`, a tool that enforces commit message guidelines to ensure consistency and quality in commit messages.
├── playground - experimental or temporary code, where developers can test new features or ideas without affecting the main codebase.
└── src - main source code directory, containing the core application code, libraries, and modules of the project.
```bash
$ make run-front-desk
```
## Credits ❤️
Then access at
[http://localhost:3000](http://localhost:3000)
### Stack
user: people
Docs is built on top of [Django Rest Framework](https://www.django-rest-framework.org/), [Next.js](https://nextjs.org/), [BlockNote.js](https://www.blocknotejs.org/), [HocusPocus](https://tiptap.dev/docs/hocuspocus/introduction) and [Yjs](https://yjs.dev/). We thank the contributors of all these projects for their awesome work!
We are proud sponsors of [BlockNotejs](https://www.blocknotejs.org/) and [Yjs](https://yjs.dev/).
password: people
### Gov ❤️ open source
Docs is the result of a joint effort led by the French 🇫🇷🥖 ([DINUM](https://www.numerique.gouv.fr/dinum/)) and German 🇩🇪🥨 governments ([ZenDiS](https://zendis.de/)).
## Contributing
We are always looking for new public partners (we are currently onboarding the Netherlands 🇳🇱🧀), feel free to [reach out](mailto:docs@numerique.gouv.fr) if you are interested in using or contributing to Docs.
This project is intended to be community-driven, so please, do not hesitate to
get in touch if you have any question related to our implementation or design
decisions.
<p align="center">
<img src="/docs/assets/europe_opensource.png" width="50%"/>
</p>
## License
This work is released under the MIT License (see [LICENSE](./LICENSE)).

View File

@@ -1,23 +0,0 @@
# Security Policy
## Reporting a Vulnerability
Security is very important to us.
If you have any issue regarding security, please disclose the information responsibly submitting [this form](https://vdp.numerique.gouv.fr/p/Send-a-report?lang=en) and not by creating an issue on the repository. You can also email us at docs@numerique.gouv.fr
We appreciate your effort to make Docs more secure.
## Vulnerability disclosure policy
Working with security issues in an open source project can be challenging, as we are required to disclose potential problems that could be exploited by attackers. With this in mind, our security fix policy is as follows:
1. The Maintainers team will handle the fix as usual (Pull Request,
release).
2. In the release notes, we will include the identification numbers from the
GitHub Advisory Database (GHSA) and, if applicable, the Common Vulnerabilities
and Exposures (CVE) identifier for the vulnerability.
3. Once this grace period has passed, we will publish the vulnerability.
By adhering to this security policy, we aim to address security concerns
effectively and responsibly in our open source software project.

View File

@@ -15,29 +15,3 @@ the following command inside your docker container:
(Note : in your development environment, you can `make migrate`.)
## [Unreleased]
## [3.3.0] - 2025-05-22
⚠️ For some advanced features (ex: Export as PDF) Docs relies on XL packages from BlockNote. These are licenced under AGPL-3.0 and are not MIT compatible. You can perfectly use Docs without these packages by setting the environment variable `PUBLISH_AS_MIT` to true. That way you'll build an image of the application without the features that are not MIT compatible. Read the [environment variables documentation](/docs/env.md) for more information.
The footer is now configurable from a customization file. To override the default one, you can
use the `THEME_CUSTOMIZATION_FILE_PATH` environment variable to point to your customization file.
The customization file must be a JSON file and must follow the rules described in the
[theming documentation](docs/theming.md).
## [3.0.0] - 2025-03-28
We are not using the nginx auth request anymore to access the collaboration server (`yProvider`)
The authentication is now managed directly from the yProvider server.
You must remove the annotation `nginx.ingress.kubernetes.io/auth-url` from the `ingressCollaborationWS`.
This means as well that the yProvider server must be able to access the Django server.
To do so, you must set the `COLLABORATION_BACKEND_BASE_URL` environment variable to the `yProvider`
service.
## [2.2.0] - 2025-02-10
- AI features are now limited to users who are authenticated. Before this release, even anonymous
users who gained editor access on a document with link reach used to get AI feature.
IF you want anonymous users to keep access on AI features, you must now define the
`AI_ALLOW_REACH_FROM` setting to "public".

View File

@@ -1,9 +1,9 @@
load('ext://uibutton', 'cmd_button', 'bool_input', 'location')
load('ext://namespace', 'namespace_create', 'namespace_inject')
namespace_create('impress')
namespace_create('desk')
docker_build(
'localhost:5001/impress-backend:latest',
'localhost:5001/people-backend:latest',
context='..',
dockerfile='../Dockerfile',
only=['./src/backend', './src/mail', './docker'],
@@ -18,41 +18,28 @@ docker_build(
)
docker_build(
'localhost:5001/impress-y-provider:latest',
'localhost:5001/people-frontend:latest',
context='..',
dockerfile='../src/frontend/servers/y-provider/Dockerfile',
only=['./src/frontend/', './docker/', './.dockerignore'],
target = 'y-provider',
dockerfile='../Dockerfile',
build_args={'ENV': 'dev'},
only=['./src/frontend', './src/mail', './docker'],
target = 'frontend-builder-dev',
live_update=[
sync('../src/frontend/servers/y-provider/src', '/home/frontend/servers/y-provider/src'),
sync('../src/frontend', '/builder'),
]
)
docker_build(
'localhost:5001/impress-frontend:latest',
context='..',
dockerfile='../src/frontend/Dockerfile',
only=['./src/frontend', './docker', './.dockerignore'],
target = 'impress',
live_update=[
sync('../src/frontend', '/home/frontend'),
]
)
k8s_resource('impress-docs-backend-migrate', resource_deps=['postgres-postgresql'])
k8s_resource('impress-docs-backend-createsuperuser', resource_deps=['impress-docs-backend-migrate'])
k8s_resource('impress-docs-backend', resource_deps=['impress-docs-backend-migrate'])
k8s_yaml(local('cd ../src/helm && helmfile -n impress -e dev template .'))
k8s_yaml(local('cd ../src/helm && helmfile -n desk -e dev template .'))
migration = '''
set -eu
# get k8s pod name from tilt resource name
POD_NAME="$(tilt get kubernetesdiscovery impress-backend -ojsonpath='{.status.pods[0].name}')"
kubectl -n impress exec "$POD_NAME" -- python manage.py makemigrations
POD_NAME="$(tilt get kubernetesdiscovery desk-backend -ojsonpath='{.status.pods[0].name}')"
kubectl -n desk exec "$POD_NAME" -- python manage.py makemigrations
'''
cmd_button('Make migration',
argv=['sh', '-c', migration],
resource='impress-backend',
resource='desk-backend',
icon_name='developer_board',
text='Run makemigration',
)
@@ -60,12 +47,12 @@ cmd_button('Make migration',
pod_migrate = '''
set -eu
# get k8s pod name from tilt resource name
POD_NAME="$(tilt get kubernetesdiscovery impress-backend -ojsonpath='{.status.pods[0].name}')"
kubectl -n impress exec "$POD_NAME" -- python manage.py migrate --no-input
POD_NAME="$(tilt get kubernetesdiscovery desk-backend -ojsonpath='{.status.pods[0].name}')"
kubectl -n desk exec "$POD_NAME" -- python manage.py migrate --no-input
'''
cmd_button('Migrate db',
argv=['sh', '-c', pod_migrate],
resource='impress-backend',
resource='desk-backend',
icon_name='developer_board',
text='Run database migration',
)

View File

@@ -6,7 +6,8 @@ REPO_DIR="$(cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd)"
UNSET_USER=0
TERRAFORM_DIRECTORY="./env.d/terraform"
COMPOSE_FILE="${REPO_DIR}/compose.yml"
COMPOSE_FILE="${REPO_DIR}/docker-compose.yml"
COMPOSE_PROJECT="people"
# _set_user: set (or unset) default user id used to run docker commands
@@ -39,8 +40,9 @@ function _set_user() {
# ARGS : docker compose command arguments
function _docker_compose() {
echo "🐳(compose) file: '${COMPOSE_FILE}'"
echo "🐳(compose) project: '${COMPOSE_PROJECT}' file: '${COMPOSE_FILE}'"
docker compose \
-p "${COMPOSE_PROJECT}" \
-f "${COMPOSE_FILE}" \
--project-directory "${REPO_DIR}" \
"$@"

102
bin/start-kind.sh Executable file → Normal file
View File

@@ -1,2 +1,102 @@
#!/bin/sh
curl https://raw.githubusercontent.com/numerique-gouv/tools/refs/heads/main/kind/create_cluster.sh | bash -s -- impress
set -o errexit
CURRENT_DIR=$(pwd)
# 0. Create ca
echo "0. Create ca"
mkcert -install
cd /tmp
mkcert "127.0.0.1.nip.io" "*.127.0.0.1.nip.io"
cd $CURRENT_DIR
# 1. Create registry container unless it already exists
echo "1. Create registry container unless it already exists"
reg_name='kind-registry'
reg_port='5001'
if [ "$(docker inspect -f '{{.State.Running}}' "${reg_name}" 2>/dev/null || true)" != 'true' ]; then
docker run \
-d --restart=always -p "127.0.0.1:${reg_port}:5000" --network bridge --name "${reg_name}" \
registry:2
fi
# 2. Create kind cluster with containerd registry config dir enabled
echo "2. Create kind cluster with containerd registry config dir enabled"
# TODO: kind will eventually enable this by default and this patch will
# be unnecessary.
#
# See:
# https://github.com/kubernetes-sigs/kind/issues/2875
# https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration
# See: https://github.com/containerd/containerd/blob/main/docs/hosts.md
cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/containerd/certs.d"
nodes:
- role: control-plane
image: kindest/node:v1.27.3
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "ingress-ready=true"
extraPortMappings:
- containerPort: 80
hostPort: 80
protocol: TCP
- containerPort: 443
hostPort: 443
protocol: TCP
- role: worker
image: kindest/node:v1.27.3
- role: worker
image: kindest/node:v1.27.3
EOF
# 3. Add the registry config to the nodes
echo "3. Add the registry config to the nodes"
#
# This is necessary because localhost resolves to loopback addresses that are
# network-namespace local.
# In other words: localhost in the container is not localhost on the host.
#
# We want a consistent name that works from both ends, so we tell containerd to
# alias localhost:${reg_port} to the registry container when pulling images
REGISTRY_DIR="/etc/containerd/certs.d/localhost:${reg_port}"
for node in $(kind get nodes); do
docker exec "${node}" mkdir -p "${REGISTRY_DIR}"
cat <<EOF | docker exec -i "${node}" cp /dev/stdin "${REGISTRY_DIR}/hosts.toml"
[host."http://${reg_name}:5000"]
EOF
done
# 4. Connect the registry to the cluster network if not already connected
echo "4. Connect the registry to the cluster network if not already connected"
# This allows kind to bootstrap the network but ensures they're on the same network
if [ "$(docker inspect -f='{{json .NetworkSettings.Networks.kind}}' "${reg_name}")" = 'null' ]; then
docker network connect "kind" "${reg_name}"
fi
# 5. Document the local registry
echo "5. Document the local registry"
# https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/generic/1755-communicating-a-local-registry
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: local-registry-hosting
namespace: kube-public
data:
localRegistryHosting.v1: |
host: "localhost:${reg_port}"
help: "https://kind.sigs.k8s.io/docs/user/local-registry/"
EOF
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
kubectl -n ingress-nginx create secret tls mkcert --key /tmp/127.0.0.1.nip.io+1-key.pem --cert /tmp/127.0.0.1.nip.io+1.pem
kubectl -n ingress-nginx patch deployments.apps ingress-nginx-controller --type 'json' -p '[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value":"--default-ssl-certificate=ingress-nginx/mkcert"}]'

View File

@@ -7,6 +7,6 @@ _dc_run \
app-dev \
python manage.py spectacular \
--api-version 'v1.0' \
--urlconf 'impress.api_urls' \
--urlconf 'people.api_urls' \
--format openapi-json \
--file /app/core/tests/swagger/swagger.json

View File

@@ -1,3 +0,0 @@
#!/bin/bash
find . -name "*.enc.*" -exec sops updatekeys -y {} \;

View File

@@ -1,28 +0,0 @@
services:
frontend:
user: "${DOCKER_USER:-1000}"
build:
context: .
dockerfile: ./src/frontend/Dockerfile
target: frontend-production
args:
API_ORIGIN: "http://localhost:8071"
PUBLISH_AS_MIT: "false"
SW_DEACTIVATED: "true"
image: impress:frontend-production
ports:
- "3000:3000"
y-provider:
user: ${DOCKER_USER:-1000}
build:
context: .
dockerfile: ./src/frontend/servers/y-provider/Dockerfile
target: y-provider
image: impress:y-provider-production
restart: unless-stopped
env_file:
- env.d/development/common
ports:
- "4444:4444"

View File

@@ -1,213 +0,0 @@
name: docs
services:
postgresql:
image: postgres:16
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 1s
timeout: 2s
retries: 300
env_file:
- env.d/development/postgresql
ports:
- "15432:5432"
redis:
image: redis:5
mailcatcher:
image: sj26/mailcatcher:latest
ports:
- "1081:1080"
minio:
user: ${DOCKER_USER:-1000}
image: minio/minio
environment:
- MINIO_ROOT_USER=impress
- MINIO_ROOT_PASSWORD=password
ports:
- '9000:9000'
- '9001:9001'
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 1s
timeout: 20s
retries: 300
entrypoint: ""
command: minio server --console-address :9001 /data
volumes:
- ./data/media:/data
createbuckets:
image: minio/mc
depends_on:
minio:
condition: service_healthy
restart: true
entrypoint: >
sh -c "
/usr/bin/mc alias set impress http://minio:9000 impress password && \
/usr/bin/mc mb impress/impress-media-storage && \
/usr/bin/mc version enable impress/impress-media-storage && \
exit 0;"
app-dev:
build:
context: .
target: backend-development
args:
DOCKER_USER: ${DOCKER_USER:-1000}
user: ${DOCKER_USER:-1000}
image: impress:backend-development
environment:
- PYLINTHOME=/app/.pylint.d
- DJANGO_CONFIGURATION=Development
env_file:
- env.d/development/common
- env.d/development/postgresql
ports:
- "8071:8000"
volumes:
- ./src/backend:/app
- ./data/static:/data/static
depends_on:
postgresql:
condition: service_healthy
restart: true
mailcatcher:
condition: service_started
redis:
condition: service_started
createbuckets:
condition: service_started
celery-dev:
user: ${DOCKER_USER:-1000}
image: impress:backend-development
command: ["celery", "-A", "impress.celery_app", "worker", "-l", "DEBUG"]
environment:
- DJANGO_CONFIGURATION=Development
env_file:
- env.d/development/common
- env.d/development/postgresql
volumes:
- ./src/backend:/app
- ./data/static:/data/static
depends_on:
- app-dev
nginx:
image: nginx:1.25
ports:
- "8083:8083"
volumes:
- ./docker/files/etc/nginx/conf.d:/etc/nginx/conf.d:ro
depends_on:
app-dev:
condition: service_started
keycloak:
condition: service_healthy
restart: true
frontend-development:
user: "${DOCKER_USER:-1000}"
build:
context: .
dockerfile: ./src/frontend/Dockerfile
target: impress-dev
args:
API_ORIGIN: "http://localhost:8071"
PUBLISH_AS_MIT: "false"
SW_DEACTIVATED: "true"
image: impress:frontend-development
volumes:
- ./src/frontend:/home/frontend
- /home/frontend/node_modules
- /home/frontend/apps/impress/node_modules
ports:
- "3000:3000"
crowdin:
image: crowdin/cli:3.16.0
volumes:
- ".:/app"
env_file:
- env.d/development/crowdin
user: "${DOCKER_USER:-1000}"
working_dir: /app
node:
image: node:22
user: "${DOCKER_USER:-1000}"
environment:
HOME: /tmp
volumes:
- ".:/app"
y-provider-development:
user: ${DOCKER_USER:-1000}
build:
context: .
dockerfile: ./src/frontend/servers/y-provider/Dockerfile
target: y-provider-development
image: impress:y-provider-development
restart: unless-stopped
env_file:
- env.d/development/common
ports:
- "4444:4444"
volumes:
- ./src/frontend/:/home/frontend
- /home/frontend/node_modules
- /home/frontend/servers/y-provider/node_modules
kc_postgresql:
image: postgres:14.3
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 1s
timeout: 2s
retries: 300
ports:
- "5433:5432"
env_file:
- env.d/development/kc_postgresql
keycloak:
image: quay.io/keycloak/keycloak:20.0.1
volumes:
- ./docker/auth/realm.json:/opt/keycloak/data/import/realm.json
command:
- start-dev
- --features=preview
- --import-realm
- --proxy=edge
- --hostname-url=http://localhost:8083
- --hostname-admin-url=http://localhost:8083/
- --hostname-strict=false
- --hostname-strict-https=false
- --health-enabled=true
- --metrics-enabled=true
healthcheck:
test: ["CMD", "curl", "--head", "-fsS", "http://localhost:8080/health/ready"]
interval: 1s
timeout: 2s
retries: 300
environment:
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: admin
KC_DB: postgres
KC_DB_URL_HOST: kc_postgresql
KC_DB_URL_DATABASE: keycloak
KC_DB_PASSWORD: pass
KC_DB_USERNAME: impress
KC_DB_SCHEMA: public
PROXY_ADDRESS_FORWARDING: 'true'
ports:
- "8080:8080"
depends_on:
kc_postgresql:
condition: service_healthy
restart: true

View File

@@ -1,7 +1,7 @@
#
# Your crowdin's credentials
#
api_token_env: CROWDIN_PERSONAL_TOKEN
api_token_env: CROWDIN_API_TOKEN
project_id_env: CROWDIN_PROJECT_ID
base_path_env: CROWDIN_BASE_PATH
@@ -14,16 +14,17 @@ preserve_hierarchy: true
#
# Files configuration
#
files: [
{
source : "/backend/locale/django.pot",
dest: "/backend-impress.pot",
translation : "/backend/locale/%locale_with_underscore%/LC_MESSAGES/django.po"
},
{
source: "/frontend/packages/i18n/locales/impress/translations-crowdin.json",
dest: "/frontend-impress.json",
translation: "/frontend/packages/i18n/locales/impress/%two_letters_code%/translations.json",
skip_untranslated_strings: true,
},
]
files:
[
{
source: "/backend/locale/django.pot",
dest: "/backend.pot",
translation: "/backend/locale/%locale_with_underscore%/LC_MESSAGES/django.po",
},
{
source: "/frontend/packages/i18n/locales/desk/translations-crowdin.json",
dest: "/desk.json",
translation: "/frontend/packages/i18n/locales/desk/%two_letters_code%/translations.json",
skip_untranslated_strings: true,
},
]

170
docker-compose.yml Normal file
View File

@@ -0,0 +1,170 @@
version: '3.8'
services:
postgresql:
image: postgres:16
env_file:
- env.d/development/postgresql
ports:
- "15432:5432"
redis:
image: redis:5
mailcatcher:
image: sj26/mailcatcher:latest
ports:
- "1081:1080"
app-dev:
build:
context: .
target: backend-development
args:
DOCKER_USER: ${DOCKER_USER:-1000}
user: ${DOCKER_USER:-1000}
image: people:backend-development
environment:
- PYLINTHOME=/app/.pylint.d
- DJANGO_CONFIGURATION=Development
env_file:
- env.d/development/common
- env.d/development/postgresql
ports:
- "8071:8000"
volumes:
- ./src/backend:/app
- ./data/media:/data/media
- ./data/static:/data/static
depends_on:
- postgresql
- mailcatcher
- redis
celery-dev:
user: ${DOCKER_USER:-1000}
image: people:backend-development
command: ["celery", "-A", "people.celery_app", "worker", "-l", "DEBUG"]
environment:
- DJANGO_CONFIGURATION=Development
env_file:
- env.d/development/common
- env.d/development/postgresql
volumes:
- ./src/backend:/app
- ./data/media:/data/media
- ./data/static:/data/static
depends_on:
- app-dev
app:
build:
context: .
target: backend-production
args:
DOCKER_USER: ${DOCKER_USER:-1000}
user: ${DOCKER_USER:-1000}
image: people:backend-production
environment:
- DJANGO_CONFIGURATION=Demo
env_file:
- env.d/development/common
- env.d/development/postgresql
volumes:
- ./data/media:/data/media
depends_on:
- postgresql
- redis
celery:
user: ${DOCKER_USER:-1000}
image: people:backend-production
command: ["celery", "-A", "people.celery_app", "worker", "-l", "INFO"]
environment:
- DJANGO_CONFIGURATION=Demo
env_file:
- env.d/development/common
- env.d/development/postgresql
depends_on:
- app
nginx:
image: nginx:1.25
ports:
- "8083:8083"
volumes:
- ./docker/files/etc/nginx/conf.d:/etc/nginx/conf.d:ro
depends_on:
- app
- keycloak
dockerize:
image: jwilder/dockerize
crowdin:
image: crowdin/cli:3.16.0
volumes:
- ".:/app"
env_file:
- env.d/development/crowdin
user: "${DOCKER_USER:-1000}"
working_dir: /app
node:
image: node:18
user: "${DOCKER_USER:-1000}"
environment:
HOME: /tmp
volumes:
- ".:/app"
terraform-state:
image: hashicorp/terraform:1.6
environment:
- TF_WORKSPACE=${PROJECT:-} # avoid env conflict in local state
user: ${DOCKER_USER:-1000}
working_dir: /app
volumes:
- ./src/terraform/create_state_bucket:/app
terraform:
image: hashicorp/terraform:1.6
user: ${DOCKER_USER:-1000}
working_dir: /app
volumes:
- ./src/terraform:/app
kc_postgresql:
image: postgres:14.3
ports:
- "5433:5432"
env_file:
- env.d/development/kc_postgresql
keycloak:
image: quay.io/keycloak/keycloak:20.0.1
volumes:
- ./docker/auth/realm.json:/opt/keycloak/data/import/realm.json
command:
- start-dev
- --features=preview
- --import-realm
- --proxy=edge
- --hostname-url=http://localhost:8083
- --hostname-admin-url=http://localhost:8083/
- --hostname-strict=false
- --hostname-strict-https=false
environment:
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: admin
KC_DB: postgres
KC_DB_URL_HOST: kc_postgresql
KC_DB_URL_DATABASE: keycloak
KC_DB_PASSWORD: pass
KC_DB_USERNAME: people
KC_DB_SCHEMA: public
PROXY_ADDRESS_FORWARDING: 'true'
ports:
- "8080:8080"
depends_on:
- kc_postgresql

View File

@@ -1,6 +1,6 @@
{
"id": "ccf4fd40-4286-474d-854a-4714282a8bec",
"realm": "impress",
"realm": "people",
"notBefore": 0,
"defaultSignatureAlgorithm": "RS256",
"revokeRefreshToken": false,
@@ -45,22 +45,22 @@
"failureFactor": 30,
"users": [
{
"username": "impress",
"email": "impress@impress.world",
"username": "people",
"email": "people@people.world",
"firstName": "John",
"lastName": "Doe",
"enabled": true,
"credentials": [
{
"type": "password",
"value": "impress"
"value": "people"
}
],
"realmRoles": ["user"]
},
{
"username": "user-e2e-chromium",
"email": "user@chromium.test",
"email": "user@chromium.e2e",
"firstName": "E2E",
"lastName": "Chromium",
"enabled": true,
@@ -74,7 +74,7 @@
},
{
"username": "user-e2e-webkit",
"email": "user@webkit.test",
"email": "user@webkit.e2e",
"firstName": "E2E",
"lastName": "Webkit",
"enabled": true,
@@ -88,7 +88,7 @@
},
{
"username": "user-e2e-firefox",
"email": "user@firefox.test",
"email": "user@firefox.e2e",
"firstName": "E2E",
"lastName": "Firefox",
"enabled": true,
@@ -114,7 +114,7 @@
},
{
"id": "1bfe401a-08fc-4d94-80e0-86c4f5195f99",
"name": "default-roles-impress",
"name": "default-roles-people",
"description": "${role_default-roles}",
"composite": true,
"composites": {
@@ -359,7 +359,7 @@
"attributes": {}
}
],
"impress": [],
"people": [],
"account": [
{
"id": "63b1a4e1-a594-4571-99c3-7c5c3efd61ce",
@@ -449,7 +449,7 @@
"groups": [],
"defaultRole": {
"id": "1bfe401a-08fc-4d94-80e0-86c4f5195f99",
"name": "default-roles-impress",
"name": "default-roles-people",
"description": "${role_default-roles}",
"composite": true,
"clientRole": false,
@@ -504,12 +504,12 @@
"clientId": "account",
"name": "${client_account}",
"rootUrl": "${authBaseUrl}",
"baseUrl": "/realms/impress/account/",
"baseUrl": "/realms/people/account/",
"surrogateAuthRequired": false,
"enabled": true,
"alwaysDisplayInConsole": false,
"clientAuthenticatorType": "client-secret",
"redirectUris": ["/realms/impress/account/*"],
"redirectUris": ["/realms/people/account/*"],
"webOrigins": [],
"notBefore": 0,
"bearerOnly": false,
@@ -546,12 +546,12 @@
"clientId": "account-console",
"name": "${client_account-console}",
"rootUrl": "${authBaseUrl}",
"baseUrl": "/realms/impress/account/",
"baseUrl": "/realms/people/account/",
"surrogateAuthRequired": false,
"enabled": true,
"alwaysDisplayInConsole": false,
"clientAuthenticatorType": "client-secret",
"redirectUris": ["/realms/impress/account/*"],
"redirectUris": ["/realms/people/account/*"],
"webOrigins": [],
"notBefore": 0,
"bearerOnly": false,
@@ -676,7 +676,7 @@
},
{
"id": "869481d0-5774-4e64-bc30-fedc7c58958f",
"clientId": "impress",
"clientId": "people",
"name": "",
"description": "",
"rootUrl": "",
@@ -791,12 +791,12 @@
"clientId": "security-admin-console",
"name": "${client_security-admin-console}",
"rootUrl": "${authAdminUrl}",
"baseUrl": "/admin/impress/console/",
"baseUrl": "/admin/people/console/",
"surrogateAuthRequired": false,
"enabled": true,
"alwaysDisplayInConsole": false,
"clientAuthenticatorType": "client-secret",
"redirectUris": ["/admin/impress/console/*"],
"redirectUris": ["/admin/people/console/*"],
"webOrigins": ["+"],
"notBefore": 0,
"bearerOnly": false,
@@ -1339,21 +1339,6 @@
"jsonType.label": "String"
}
},
{
"id": "qb109597-e31e-46d7-7844-62e5fcf32ac8",
"name": "email sub",
"protocol": "openid-connect",
"protocolMapper": "oidc-usermodel-property-mapper",
"consentRequired": false,
"config": {
"userinfo.token.claim": "true",
"user.attribute": "email",
"id.token.claim": "true",
"access.token.claim": "true",
"claim.name": "sub",
"jsonType.label": "String"
}
},
{
"id": "61c135e5-2447-494b-bc70-9612f383be27",
"name": "email verified",

View File

@@ -4,49 +4,10 @@ server {
server_name localhost;
charset utf-8;
# Proxy auth for media
location /media/ {
# Auth request configuration
auth_request /media-auth;
auth_request_set $authHeader $upstream_http_authorization;
auth_request_set $authDate $upstream_http_x_amz_date;
auth_request_set $authContentSha256 $upstream_http_x_amz_content_sha256;
# Pass specific headers from the auth response
proxy_set_header Authorization $authHeader;
proxy_set_header X-Amz-Date $authDate;
proxy_set_header X-Amz-Content-SHA256 $authContentSha256;
# Get resource from Minio
proxy_pass http://minio:9000/impress-media-storage/;
proxy_set_header Host minio:9000;
add_header Content-Security-Policy "default-src 'none'" always;
}
location /media-auth {
proxy_pass http://app-dev:8000/api/v1.0/documents/media-auth/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Original-URL $request_uri;
# Prevent the body from being passed
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-Method $request_method;
}
location / {
proxy_pass http://keycloak:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Increase proxy buffer size to allow keycloak to send large
# header responses when a user is created.
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
}
}

View File

@@ -1,112 +0,0 @@
upstream docs_backend {
server ${BACKEND_HOST}:8000 fail_timeout=0;
}
upstream docs_frontend {
server ${FRONTEND_HOST}:3000 fail_timeout=0;
}
server {
listen 8083;
server_name localhost;
charset utf-8;
# Disables server version feedback on pages and in headers
server_tokens off;
proxy_ssl_server_name on;
location @proxy_to_docs_backend {
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
proxy_pass http://docs_backend;
}
location @proxy_to_docs_frontend {
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
proxy_pass http://docs_frontend;
}
location / {
try_files $uri @proxy_to_docs_frontend;
}
location /api {
try_files $uri @proxy_to_docs_backend;
}
location /admin {
try_files $uri @proxy_to_docs_backend;
}
location /static {
try_files $uri @proxy_to_docs_backend;
}
# Proxy auth for collaboration server
location /collaboration/ws/ {
# Ensure WebSocket upgrade
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
# Collaboration server
proxy_pass http://${YPROVIDER_HOST}:4444;
# Set appropriate timeout for WebSocket
proxy_read_timeout 86400;
proxy_send_timeout 86400;
# Preserve original host and additional headers
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Origin $http_origin;
proxy_set_header Host $host;
}
location /collaboration/api/ {
# Collaboration server
proxy_pass http://${YPROVIDER_HOST}:4444;
proxy_set_header Host $host;
}
# Proxy auth for media
location /media/ {
# Auth request configuration
auth_request /media-auth;
auth_request_set $authHeader $upstream_http_authorization;
auth_request_set $authDate $upstream_http_x_amz_date;
auth_request_set $authContentSha256 $upstream_http_x_amz_content_sha256;
# Pass specific headers from the auth response
proxy_set_header Authorization $authHeader;
proxy_set_header X-Amz-Date $authDate;
proxy_set_header X-Amz-Content-SHA256 $authContentSha256;
# Get resource from Minio
proxy_pass https://${S3_HOST}/${BUCKET_NAME}/;
proxy_set_header Host ${S3_HOST};
proxy_ssl_name ${S3_HOST};
add_header Content-Security-Policy "default-src 'none'" always;
}
location /media-auth {
proxy_pass http://docs_backend/api/v1.0/documents/media-auth/;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Original-URL $request_uri;
# Prevent the body from being passed
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-Method $request_method;
}
}

View File

@@ -13,7 +13,7 @@
#
# To pass environment variables, you can either use the -e option of the docker run command:
#
# docker run --rm -e USER_NAME=foo -e HOME='/home/foo' impress:latest python manage.py migrate
# docker run --rm -e USER_NAME=foo -e HOME='/home/foo' people:latest python manage.py migrate
#
# or define new variables in an environment file to use with docker or docker compose:
#
@@ -21,7 +21,7 @@
# USER_NAME=foo
# HOME=/home/foo
#
# docker run --rm --env-file env.d/production impress:latest python manage.py migrate
# docker run --rm --env-file env.d/production people:latest python manage.py migrate
#
echo "🐳(entrypoint) creating user running in the container..."

View File

@@ -1,6 +1,6 @@
# Gunicorn-django settings
bind = ["0.0.0.0:8000"]
name = "impress"
name = "people"
python_path = "/app"
# Run

View File

@@ -1,193 +0,0 @@
## Decision TLDR;
We will use Yjs a CRDT-based library for the collaborative editing of the documents.
## Status
Accepted
## Context
We need to implement a collaborative editing feature for the documents that supports real-time collaboration, offline capabilities, and seamless integration with our Django backend.
## Considered alternatives
### ProseMirror
A robust toolkit for building rich-text editors with collaboration capabilities.
| Pros | Cons |
| --- | --- |
| Mature ecosystem | Complex integration with Django |
| Rich text editing features | Steeper learning curve |
| Used by major companies | More complex to implement offline support |
| Large community | |
### ShareDB
Real-time database backend based on Operational Transformation.
| Pros | Cons |
| --- | --- |
| Battle-tested in production | Complex setup required |
| Strong consistency model | Requires specific backend architecture |
| Good documentation | Less flexible with different backends |
| | Higher latency compared to CRDTs |
### Convergence
Complete enterprise solution for real-time collaboration.
| Pros | Cons |
| --- | --- |
| Full-featured solution | Commercial licensing |
| Built-in presence features | Less community support |
| Enterprise support | More expensive |
| Good offline support | Overkill for basic needs |
### CRDT-based Solutions Comparison
A CRDT-based library specifically designed for real-time collaboration.
| Category | Pros | Cons |
|----------|------|------|
| Technical Implementation | • Native real-time collaboration<br>• No central conflict resolution needed<br>• Works well with Django backend<br>• Automatic state synchronization | • Learning curve for CRDT concepts<br>• More complex initial setup<br>• Additional metadata overhead |
| User Experience | • Instant local updates<br>• Works offline by default<br>• Low latency<br>• Smooth concurrent editing | • Eventual consistency might cause brief inconsistencies<br>• UI must handle temporary conflicts |
| Performance | • Excellent scaling with multiple users<br>• Reduced server load<br>• Efficient network usage<br>• Good memory optimization (especially Yjs) | • Slightly higher memory usage<br>• Initial state sync can be larger |
| Development | • No need to build conflict resolution<br>• Simple integration with text editors<br>• Future-proof architecture | • Team needs to learn new concepts<br>• Fewer ready-made solutions<br>• May need to build some features from scratch |
| Maintenance | • Less server infrastructure<br>• Simpler deployment<br>• Fewer points of failure | • Debugging can be more complex<br>• State management requires careful handling |
| Business Impact | • Better offline support for users<br>• Scales well as user base grows<br>• No licensing costs (with Yjs) | • Initial development time might be longer<br>• Team training required |
#### Yjs
- **Type**: State-based CRDT
- **Implementation**: JavaScript/TypeScript
- **Features**:
- Rich text collaboration
- Shared types (Array, Map, XML)
- Binary encoding
- P2P support
- **Performance**: Excellent for text editing
- **Memory Usage**: Optimized
- **License**: MIT
#### Automerge
- **Type**: Operation-based CRDT
- **Implementation**: JavaScript/Rust
- **Features**:
- JSON-like data structures
- Change history
- Undo/Redo
- Binary format
- **Performance**: Good, with Rust backend
- **Memory Usage**: Higher than Yjs
- **License**: MIT
#### Legion
- **Type**: State-based CRDT
- **Implementation**: Rust with JS bindings
- **Features**:
- High performance
- Memory efficient
- Binary protocol
- **Performance**: Excellent
- **Memory Usage**: Very efficient
- **License**: Apache 2.0
#### Diamond Types
- **Type**: Operation-based CRDT
- **Implementation**: TypeScript
- **Features**:
- Specialized for text
- Small memory footprint
- Simple API
- **Performance**: Good for text
- **Memory Usage**: Efficient
- **License**: MIT
Comparison Table:
| Feature | Yjs | Automerge | Legion | Diamond Types |
|---------|-----|-----------|--------|---------------|
| Text Editing | ✅ Excellent | ✅ Good | ⚠️ Basic | ✅ Excellent |
| Structured Data | ✅ | ✅ | ✅ | ⚠️ |
| Memory Efficiency | ✅ High | ⚠️ Medium | ✅ Very High | ✅ High |
| Network Efficiency | ✅ | ⚠️ | ✅ | ✅ |
| Maturity | ✅ | ✅ | ⚠️ | ⚠️ |
| Community Size | ✅ Large | ✅ Large | ⚠️ Small | ⚠️ Small |
| Documentation | ✅ | ✅ | ⚠️ | ⚠️ |
| Backend Options | ✅ Many | ✅ Many | ⚠️ Limited | ⚠️ Limited |
Key Differences:
1. **Implementation Approach**:
- Yjs: Optimized for text and rich-text editing
- Automerge: General-purpose JSON CRDT
- Legion: Performance-focused with Rust
- Diamond Types: Specialized for text collaboration
2. **Performance Characteristics**:
- Yjs: Best for text editing scenarios
- Automerge: Good all-around performance
- Legion: Excellent raw performance
- Diamond Types: Optimized for text
3. **Ecosystem Integration**:
- Yjs: Wide range of integrations
- Automerge: Good JavaScript ecosystem
- Legion: Limited but growing
- Diamond Types: Focused on text editors
This analysis reinforces our choice of Yjs for the CRDT-based option as it provides:
- Best-in-class text editing performance
- Mature ecosystem
- Active community
- Excellent documentation
- Wide range of backend options
## Decision
After evaluating the alternatives, we choose Yjs for the following reasons:
1. **Technical Fit:**
- Native CRDT support ensures reliable collaboration
- Excellent offline capabilities
- Good performance characteristics
- Flexible backend integration options
2. **Project Requirements Match:**
- Easy integration with our Django backend
- Supports our core collaborative features
- Manageable learning curve for the team
3. **Community & Support:**
- Active development
- Growing community
- Good documentation
- Open source with MIT license
### Comparison of Key Features:
| Feature | Yjs (CRDT) | ProseMirror | ShareDB | Convergence |
|---------|-----|-------------|----------|-------------|
| Real-time Collaboration | ✅ | ✅ | ✅ | ✅ |
| Offline Support | ✅ | ⚠️ | ⚠️ | ✅ |
| Django Integration | Easy | Complex | Complex | Moderate |
| Learning Curve | Medium | High | High | Medium |
| Cost | Free | Free | Free | Paid |
| Community Size | Growing | Large | Medium | Small |
## Consequences
### Positive
- Simplified implementation of real-time collaboration
- Good developer experience
- Future-proof technology choice
- No licensing costs
### Negative
- Team needs to learn CRDT concepts
- Newer technology compared to alternatives
- May need to build some features available out-of-the-box in other solutions
### Risks
- Community support might not grow as expected
- May discover limitations as we scale

View File

@@ -1,19 +0,0 @@
## Architecture
### Global system architecture
```mermaid
flowchart TD
User -- HTTP --> Front("Frontend (NextJS SPA)")
Front -- REST API --> Back("Backend (Django)")
Front -- WebSocket --> Yserver("Microservice Yjs (Express)") -- WebSocket --> CollaborationServer("Collaboration server (Hocuspocus)") -- REST API <--> Back
Front -- OIDC --> Back -- OIDC ---> OIDC("Keycloak / ProConnect")
Back -- REST API --> Yserver
Back --> DB("Database (PostgreSQL)")
Back <--> Celery --> DB
Back ----> S3("Minio (S3)")
```
### Architecture decision records
- [ADR-0001-20250106-use-yjs-for-docs-editing](./adr/ADR-0001-20250106-use-yjs-for-docs-editing.md)

Binary file not shown.

Before

Width:  |  Height:  |  Size: 215 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.8 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 80 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.3 KiB

View File

@@ -1,144 +0,0 @@
# Docs variables
Here we describe all environment variables that can be set for the docs application.
## impress-backend container
These are the environment variables you can set for the `impress-backend` container.
| Option | Description | default |
| ----------------------------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------- |
| AI_ALLOW_REACH_FROM | Users that can use AI must be this level. options are "public", "authenticated", "restricted" | authenticated |
| AI_API_KEY | AI key to be used for AI Base url | |
| AI_BASE_URL | OpenAI compatible AI base url | |
| AI_FEATURE_ENABLED | Enable AI options | false |
| AI_MODEL | AI Model to use | |
| ALLOW_LOGOUT_GET_METHOD | Allow get logout method | true |
| API_USERS_LIST_LIMIT | Limit on API users | 5 |
| API_USERS_LIST_THROTTLE_RATE_BURST | throttle rate for api on burst | 30/minute |
| API_USERS_LIST_THROTTLE_RATE_SUSTAINED | throttle rate for api | 180/hour |
| AWS_S3_ACCESS_KEY_ID | access id for s3 endpoint | |
| AWS_S3_ENDPOINT_URL | S3 endpoint | |
| AWS_S3_REGION_NAME | region name for s3 endpoint | |
| AWS_S3_SECRET_ACCESS_KEY | access key for s3 endpoint | |
| AWS_STORAGE_BUCKET_NAME | bucket name for s3 endpoint | impress-media-storage |
| CACHES_DEFAULT_TIMEOUT | cache default timeout | 30 |
| CACHES_KEY_PREFIX | The prefix used to every cache keys. | docs |
| COLLABORATION_API_URL | collaboration api host | |
| COLLABORATION_SERVER_SECRET | collaboration api secret | |
| COLLABORATION_WS_NOT_CONNECTED_READY_ONLY | Users not connected to the collaboration server cannot edit | false |
| COLLABORATION_WS_URL | collaboration websocket url | |
| CONVERSION_API_CONTENT_FIELD | Conversion api content field | content |
| CONVERSION_API_ENDPOINT | Conversion API endpoint | convert-markdown |
| CONVERSION_API_SECURE | Require secure conversion api | false |
| CONVERSION_API_TIMEOUT | Conversion api timeout | 30 |
| CONTENT_SECURITY_POLICY_DIRECTIVES | A dict of directives set in the Content-Security-Policy header | All directives are set to 'none' |
| CONTENT_SECURITY_POLICY_EXCLUDE_URL_PREFIXES | Url with this prefix will not have the header Content-Security-Policy included | |
| CRISP_WEBSITE_ID | crisp website id for support | |
| DB_ENGINE | engine to use for database connections | django.db.backends.postgresql_psycopg2 |
| DB_HOST | host of the database | localhost |
| DB_NAME | name of the database | impress |
| DB_PASSWORD | password to authenticate with | pass |
| DB_PORT | port of the database | 5432 |
| DB_USER | user to authenticate with | dinum |
| DJANGO_ALLOWED_HOSTS | allowed hosts | [] |
| DJANGO_CELERY_BROKER_TRANSPORT_OPTIONS | celery broker transport options | {} |
| DJANGO_CELERY_BROKER_URL | celery broker url | redis://redis:6379/0 |
| DJANGO_CORS_ALLOW_ALL_ORIGINS | allow all CORS origins | false |
| DJANGO_CORS_ALLOWED_ORIGIN_REGEXES | list of origins allowed for CORS using regulair expressions | [] |
| DJANGO_CORS_ALLOWED_ORIGINS | list of origins allowed for CORS | [] |
| DJANGO_CSRF_TRUSTED_ORIGINS | CSRF trusted origins | [] |
| DJANGO_EMAIL_BACKEND | email backend library | django.core.mail.backends.smtp.EmailBackend |
| DJANGO_EMAIL_BRAND_NAME | brand name for email | |
| DJANGO_EMAIL_FROM | email address used as sender | from@example.com |
| DJANGO_EMAIL_HOST | host name of email | |
| DJANGO_EMAIL_HOST_PASSWORD | password to authenticate with on the email host | |
| DJANGO_EMAIL_HOST_USER | user to authenticate with on the email host | |
| DJANGO_EMAIL_LOGO_IMG | logo for the email | |
| DJANGO_EMAIL_PORT | port used to connect to email host | |
| DJANGO_EMAIL_USE_SSL | use sstl for email host connection | false |
| DJANGO_EMAIL_USE_TLS | use tls for email host connection | false |
| DJANGO_SECRET_KEY | secret key | |
| DJANGO_SERVER_TO_SERVER_API_TOKENS | | [] |
| DOCUMENT_IMAGE_MAX_SIZE | maximum size of document in bytes | 10485760 |
| FRONTEND_CSS_URL | To add a external css file to the app | |
| FRONTEND_HOMEPAGE_FEATURE_ENABLED | frontend feature flag to display the homepage | false |
| FRONTEND_THEME | frontend theme to use | |
| LANGUAGE_CODE | default language | en-us |
| LOGGING_LEVEL_LOGGERS_APP | application logging level. options are "DEBUG", "INFO", "WARN", "ERROR", "CRITICAL" | INFO |
| LOGGING_LEVEL_LOGGERS_ROOT | default logging level. options are "DEBUG", "INFO", "WARN", "ERROR", "CRITICAL" | INFO |
| LOGIN_REDIRECT_URL | login redirect url | |
| LOGIN_REDIRECT_URL_FAILURE | login redirect url on failure | |
| LOGOUT_REDIRECT_URL | logout redirect url | |
| MALWARE_DETECTION_BACKEND | The malware detection backend use from the django-lasuite package | lasuite.malware_detection.backends.dummy.DummyBackend |
| MALWARE_DETECTION_PARAMETERS | A dict containing all the parameters to initiate the malware detection backend | {"callback_path": "core.malware_detection.malware_detection_callback",} |
| MEDIA_BASE_URL | | |
| OIDC_ALLOW_DUPLICATE_EMAILS | Allow duplicate emails | false |
| OIDC_AUTH_REQUEST_EXTRA_PARAMS | OIDC extra auth parameters | {} |
| OIDC_CREATE_USER | create used on OIDC | false |
| OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION | faillback to email for identification | true |
| OIDC_OP_AUTHORIZATION_ENDPOINT | Authorization endpoint for OIDC | |
| OIDC_OP_JWKS_ENDPOINT | JWKS endpoint for OIDC | |
| OIDC_OP_LOGOUT_ENDPOINT | Logout endpoint for OIDC | |
| OIDC_OP_TOKEN_ENDPOINT | Token endpoint for OIDC | |
| OIDC_OP_USER_ENDPOINT | User endpoint for OIDC | |
| OIDC_REDIRECT_ALLOWED_HOSTS | Allowed hosts for OIDC redirect url | [] |
| OIDC_REDIRECT_REQUIRE_HTTPS | Require https for OIDC redirect url | false |
| OIDC_RP_CLIENT_ID | client id used for OIDC | impress |
| OIDC_RP_CLIENT_SECRET | client secret used for OIDC | |
| OIDC_RP_SCOPES | scopes requested for OIDC | openid email |
| OIDC_RP_SIGN_ALGO | verification algorithm used OIDC tokens | RS256 |
| OIDC_STORE_ID_TOKEN | Store OIDC token | true |
| OIDC_USE_NONCE | use nonce for OIDC | true |
| OIDC_USERINFO_FULLNAME_FIELDS | OIDC token claims to create full name | ["first_name", "last_name"] |
| OIDC_USERINFO_SHORTNAME_FIELD | OIDC token claims to create shortname | first_name |
| POSTHOG_KEY | posthog key for analytics | |
| REDIS_URL | cache url | redis://redis:6379/1 |
| SENTRY_DSN | sentry host | |
| SESSION_COOKIE_AGE | duration of the cookie session | 60*60*12 |
| SPECTACULAR_SETTINGS_ENABLE_DJANGO_DEPLOY_CHECK | | false |
| STORAGES_STATICFILES_BACKEND | | whitenoise.storage.CompressedManifestStaticFilesStorage |
| THEME_CUSTOMIZATION_CACHE_TIMEOUT | Cache duration for the customization settings | 86400 |
| THEME_CUSTOMIZATION_FILE_PATH | full path to the file customizing the theme. An example is provided in src/backend/impress/configuration/theme/default.json | BASE_DIR/impress/configuration/theme/default.json |
| TRASHBIN_CUTOFF_DAYS | trashbin cutoff | 30 |
| USER_OIDC_ESSENTIAL_CLAIMS | essential claims in OIDC token | [] |
| Y_PROVIDER_API_BASE_URL | Y Provider url | |
| Y_PROVIDER_API_KEY | Y provider API key | |
## impress-frontend image
These are the environment variables you can set to build the `impress-frontend` image.
Depending on how you are building the front-end application, this variable is used in different ways.
If you want to build the Docker image, this variable is used as an argument in the build command.
Example:
```
docker build -f src/frontend/Dockerfile --target frontend-production --build-arg PUBLISH_AS_MIT=false docs-frontend:latest
```
If you want to build the front-end application using the yarn build command, you can edit the file `src/frontend/apps/impress/.env` with the `NODE_ENV=production` environment variable and modify it. Alternatively, you can use the listed environment variables with the prefix `NEXT_PUBLIC_` (for example, `NEXT_PUBLIC_PUBLISH_AS_MIT=false`).
Example:
```
cd src/frontend/apps/impress
NODE_ENV=production NEXT_PUBLIC_PUBLISH_AS_MIT=false yarn build
```
| Option | Description | default |
| ----------------------------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------- |
| API_ORIGIN | backend domain - it uses the current domain if not initialized | |
| SW_DEACTIVATED | To not install the service worker | |
| PUBLISH_AS_MIT | Removes packages whose licences are incompatible with the MIT licence (see below) | true |
Packages with licences incompatible with the MIT licence:
* `xl-docx-exporter`: [AGPL-3.0](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-docx-exporter/LICENSE),
* `xl-pdf-exporter`: [AGPL-3.0](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-pdf-exporter/LICENSE)
In `.env.development`, `PUBLISH_AS_MIT` is set to `false`, allowing developers to test Docs with all its features.
⚠️ If you run Docs in production with `PUBLISH_AS_MIT` set to `false` make sure you fulfill your [BlockNote licensing](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-pdf-exporter/LICENSE) or [subscription](https://www.blocknotejs.org/about#partner-with-us) obligations.

View File

@@ -1,78 +0,0 @@
services:
postgresql:
image: postgres:16
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 1s
timeout: 2s
retries: 300
env_file:
- env.d/postgresql
- env.d/common
environment:
- PGDATA=/var/lib/postgresql/data/pgdata
volumes:
- ./data/databases/backend:/var/lib/postgresql/data/pgdata
redis:
image: redis:8
backend:
image: lasuite/impress-backend:latest
user: ${DOCKER_USER:-1000}
restart: always
environment:
- DJANGO_CONFIGURATION=Production
env_file:
- env.d/common
- env.d/backend
- env.d/yprovider
- env.d/postgresql
healthcheck:
test: ["CMD", "python", "manage.py", "check"]
interval: 15s
timeout: 30s
retries: 20
start_period: 10s
depends_on:
postgresql:
condition: service_healthy
restart: true
redis:
condition: service_started
y-provider:
image: lasuite/impress-y-provider:latest
user: ${DOCKER_USER:-1000}
env_file:
- env.d/common
- env.d/yprovider
frontend:
image: lasuite/impress-frontend:latest
user: "101"
entrypoint:
- /docker-entrypoint.sh
command: ["nginx", "-g", "daemon off;"]
env_file:
- env.d/common
# Uncomment and set your values if using our nginx proxy example
#environment:
# - VIRTUAL_HOST=${DOCS_HOST} # used by nginx proxy
# - VIRTUAL_PORT=8083 # used by nginx proxy
# - LETSENCRYPT_HOST=${DOCS_HOST} # used by lets encrypt to generate TLS certificate
volumes:
- ./default.conf.template:/etc/nginx/templates/docs.conf.template
depends_on:
backend:
condition: service_healthy
# Uncomment if using our nginx proxy example
# networks:
# - proxy-tier
# - default
# Uncomment if using our nginx proxy example
#networks:
# proxy-tier:
# external: true

View File

@@ -1,88 +0,0 @@
# Deploy and Configure Keycloak for Docs
## Installation
> \[!CAUTION\]
> We provide those instructions as an example, for production environments, you should follow the [official documentation](https://www.keycloak.org/documentation).
### Step 1: Prepare your working environment:
```bash
mkdir keycloak
curl -o compose.yaml https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/docs/examples/compose/keycloak/compose.yaml
curl -o env.d/kc_postgresql https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/env.d/production.dist/kc_postgresql
curl -o env.d/keycloak https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/env.d/production.dist/keycloak
```
### Step 2:. Update `env.d/` files
The following variables need to be updated with your own values, others can be left as is:
```env
POSTGRES_PASSWORD=<generate postgres password>
KC_HOSTNAME=https://id.yourdomain.tld # Change with your own URL
KC_BOOTSTRAP_ADMIN_PASSWORD=<generate your password>
```
### Step 3: Expose keycloak instance on https
> \[!NOTE\]
> You can skip this section if you already have your own setup.
To access your Keycloak instance on the public network, it needs to be exposed on a domain with SSL termination. You can use our [example with nginx proxy and Let's Encrypt companion](../nginx-proxy/README.md) for automated creation/renewal of certificates using [acme.sh](http://acme.sh).
If following our example, uncomment the environment and network sections in compose file and update it with your values.
```yaml
version: '3'
services:
keycloak:
...
# Uncomment and set your values if using our nginx proxy example
# environment:
# - VIRTUAL_HOST=id.yourdomain.tld # used by nginx proxy
# - VIRTUAL_PORT=8080 # used by nginx proxy
# - LETSENCRYPT_HOST=id.yourdomain.tld # used by lets encrypt to generate TLS certificate
...
# Uncomment if using our nginx proxy example
# networks:
# - proxy-tier
# - default
# Uncomment if using our nginx proxy example
#networks:
# proxy-tier:
# external: true
```
### Step 4: Start the service
```bash
`docker compose up -d`
```
Your keycloak instance is now available on https://doc.yourdomain.tld
## Creating an OIDC Client for Docs Application
### Step 1: Create a New Realm
1. Log in to the Keycloak administration console.
2. Navigate to the realm tab and click on the "Create realm" button.
3. Enter the name of the realm - `docs`.
4. Click "Create".
#### Step 2: Create a New Client
1. Navigate to the "Clients" tab.
2. Click on the "Create client" button.
3. Enter the client ID - e.g. `docs`.
4. Enable "Client authentication" option.
6. Set the "Valid redirect URIs" to the URL of your docs application suffixed with `/*` - e.g., "https://docs.example.com/*".
1. Set the "Web Origins" to the URL of your docs application - e.g. `https://docs.example.com`.
1. Click "Save".
#### Step 3: Get Client Credentials
1. Go to the "Credentials" tab.
2. Copy the client ID (`docs` in this example) and the client secret.

View File

@@ -1,36 +0,0 @@
services:
kc_postgresql:
image: postgres:16
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 1s
timeout: 2s
retries: 300
env_file:
- env.d/kc_postgresql
volumes:
- ./data/keycloak:/var/lib/postgresql/data/pgdata
keycloak:
image: quay.io/keycloak/keycloak:26.1.3
command: ["start"]
env_file:
- env.d/kc_postgresql
- env.d/keycloak
# Uncomment and set your values if using our nginx proxy example
# environment:
# - VIRTUAL_HOST=id.yourdomain.tld # used by nginx proxy
# - VIRTUAL_PORT=8080 # used by nginx proxy
# - LETSENCRYPT_HOST=id.yourdomain.tld # used by lets encrypt to generate TLS certificate
depends_on:
kc_postgresql::
condition: service_healthy
restart: true
# Uncomment if using our nginx proxy example
# networks:
# - proxy-tier
# - default
#
#networks:
# proxy-tier:
# external: true

View File

@@ -1,103 +0,0 @@
# Deploy and Configure Minio for Docs
## Installation
> \[!CAUTION\]
> We provide those instructions as an example, it should not be run in production. For production environments, deploy MinIO [in a Multi-Node Multi-Drive (Distributed)](https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html#minio-mnmd) topology
### Step 1: Prepare your working environment:
```bash
mkdir minio
curl -o compose.yaml https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/docs/examples/compose/minio/compose.yaml
```
### Step 2:. Update compose file with your own values
```yaml
version: '3'
services:
minio:
...
environment:
- MINIO_ROOT_USER=<Set minio root username>
- MINIO_ROOT_PASSWORD=<Set minio root password>
```
### Step 3: Expose MinIO instance
#### Option 1: Internal network
You may not need to expose your MinIO instance to the public if only services hosted on the same private network need to access to your MinIO instance.
You should create a docker network that will be shared between those services
```bash
docker network create storage-tier
```
#### Option 2: Public network
If you want to expose your MinIO instance to the public, it needs to be exposed on a domain with SSL termination. You can use our [example](../nginx-proxy/README.md) with an nginx proxy and Let's Encrypt companion for automated creation/renewal of Let's Encrypt certificates using [acme.sh](http://acme.sh).
If following our example, uncomment the environment and network sections in compose file and update it with your values.
```yaml
version: '3'
services:
docs:
...
minio:
...
environment:
...
# - VIRTUAL_HOST=storage.yourdomain.tld # used by nginx proxy
# - VIRTUAL_PORT=9000 # used by nginx proxy
# - LETSENCRYPT_HOST=storage.yourdomain.tld # used by lets encrypt to generate TLS certificate
...
# Uncomment if using our nginx proxy example
# networks:
# - proxy-tier
# - default
# Uncomment if using our nginx proxy example
#networks:
# proxy-tier:
# external: true
```
In this example we are only exposing MinIO API service. Follow the official documentation to configure Minio WebUI.
### Step 4: Start the service
```bash
`docker compose up -d`
```
Your minio instance is now available on https://storage.yourdomain.tld
## Creating a user and bucket for your Docs instance
### Installing mc
Follow the [official documentation](https://min.io/docs/minio/linux/reference/minio-mc.html#install-mc) to install mc
### Step 1: Configure `mc` to connect to your MinIO Server with your root user
```shellscript
mc alias set minio <MINIO_SERVER_URL> <MINIO_ROOT_USER> <MINIO_ROOT_PASSWORD>
```
Replace the values with those you have set in the previous steps
### Step 2: Create a new bucket with versioning enabled
```shellscript
mc mb --with-versioning minio/<your-bucket-name>
```
Replace `your-bucket-name` with the desired name for your bucket e.g. `docs-media-storage`
### Additional notes:
For increased security you should create a dedicated user with `readwrite` access to the Bucket. In the following example we will use MinIO root user.

View File

@@ -1,27 +0,0 @@
services:
minio:
image: minio/minio
environment:
- MINIO_ROOT_USER=<set minio root username>
- MINIO_ROOT_PASSWORD=<set minio root password>
# Uncomment and set your values if using our nginx proxy example
# - VIRTUAL_HOST=storage.yourdomain.tld # used by nginx proxy
# - VIRTUAL_PORT=9000 # used by nginx proxy
# - LETSENCRYPT_HOST=storage.yourdomain.tld # used by lets encrypt to generate TLS certificate
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 1s
timeout: 20s
retries: 300
entrypoint: ""
command: minio server /data
volumes:
- ./data/minio:/data
# Uncomment if using our nginx proxy example
# networks:
# - proxy-tier
# Uncomment if using our nginx proxy example
#networks:
# proxy-tier:
# external: true

View File

@@ -1,39 +0,0 @@
# Nginx proxy with automatic SSL certificates
> \[!CAUTION\]
> We provide those instructions as an example, for extended development or production environments, you should follow the [official documentation](https://github.com/nginx-proxy/acme-companion/tree/main/docs).
Nginx-proxy sets up a container running nginx and docker-gen. docker-gen generates reverse proxy configs for nginx and reloads nginx when containers are started and stopped.
Acme-companion is a lightweight companion container for nginx-proxy. It handles the automated creation, renewal and use of SSL certificates for proxied Docker containers through the ACME protocol.
## Installation
### Step 1: Prepare your working environment:
```bash
mkdir nginx-proxy
curl -o compose.yaml https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/docs/examples/compose/nginx-proxy/compose.yaml
```
### Step 2: Edit `DEFAULT_EMAIL` in the compose file.
Albeit optional, it is recommended to provide a valid default email address through the `DEFAULT_EMAIL` environment variable, so that Let's Encrypt can warn you about expiring certificates and allow you to recover your account.
### Step 3: Create docker network
Containers need share the same network for auto-discovery.
```bash
docker network create proxy-tier
```
### Step 4: Start service
```bash
docker compose up -d
```
## Usage
Once both nginx-proxy and acme-companion containers are up and running, start any container you want proxied with environment variables `VIRTUAL_HOST` and `LETSENCRYPT_HOST` both set to the domain(s) your proxied container is going to use.

View File

@@ -1,36 +0,0 @@
services:
nginx-proxy:
image: nginxproxy/nginx-proxy
container_name: nginx-proxy
ports:
- "80:80"
- "443:443"
volumes:
- html:/usr/share/nginx/html
- certs:/etc/nginx/certs:ro
- /var/run/docker.sock:/tmp/docker.sock:ro
networks:
- proxy-tier
acme-companion:
image: nginxproxy/acme-companion
container_name: nginx-proxy-acme
environment:
- DEFAULT_EMAIL=mail@yourdomain.tld
volumes_from:
- nginx-proxy
volumes:
- certs:/etc/nginx/certs:rw
- acme:/etc/acme.sh
- /var/run/docker.sock:/var/run/docker.sock:ro
networks:
- proxy-tier
networks:
proxy-tier:
external: true
volumes:
html:
certs:
acme:

View File

@@ -1,176 +0,0 @@
image:
repository: lasuite/impress-backend
pullPolicy: Always
tag: "latest"
backend:
replicas: 1
envVars:
COLLABORATION_API_URL: https://impress.127.0.0.1.nip.io/collaboration/api/
COLLABORATION_SERVER_SECRET: my-secret
DJANGO_CSRF_TRUSTED_ORIGINS: https://impress.127.0.0.1.nip.io
DJANGO_CONFIGURATION: Feature
DJANGO_ALLOWED_HOSTS: impress.127.0.0.1.nip.io
DJANGO_SERVER_TO_SERVER_API_TOKENS: secret-api-key
DJANGO_SECRET_KEY: AgoodOrAbadKey
DJANGO_SETTINGS_MODULE: impress.settings
DJANGO_SUPERUSER_PASSWORD: admin
DJANGO_EMAIL_BRAND_NAME: "La Suite Numérique"
DJANGO_EMAIL_HOST: "mailcatcher"
DJANGO_EMAIL_LOGO_IMG: https://impress.127.0.0.1.nip.io/assets/logo-suite-numerique.png
DJANGO_EMAIL_PORT: 1025
DJANGO_EMAIL_USE_SSL: False
LOGGING_LEVEL_HANDLERS_CONSOLE: ERROR
LOGGING_LEVEL_LOGGERS_ROOT: INFO
LOGGING_LEVEL_LOGGERS_APP: INFO
OIDC_OP_JWKS_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/certs
OIDC_OP_AUTHORIZATION_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/token
OIDC_OP_USER_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/userinfo
OIDC_OP_LOGOUT_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/session/end
OIDC_RP_CLIENT_ID: impress
OIDC_RP_CLIENT_SECRET: ThisIsAnExampleKeyForDevPurposeOnly
OIDC_RP_SIGN_ALGO: RS256
OIDC_RP_SCOPES: "openid email"
OIDC_VERIFY_SSL: False
OIDC_USERINFO_SHORTNAME_FIELD: "given_name"
OIDC_USERINFO_FULLNAME_FIELDS: "given_name,usual_name"
OIDC_REDIRECT_ALLOWED_HOSTS: https://impress.127.0.0.1.nip.io
OIDC_AUTH_REQUEST_EXTRA_PARAMS: "{'acr_values': 'eidas1'}"
LOGIN_REDIRECT_URL: https://impress.127.0.0.1.nip.io
LOGIN_REDIRECT_URL_FAILURE: https://impress.127.0.0.1.nip.io
LOGOUT_REDIRECT_URL: https://impress.127.0.0.1.nip.io
POSTHOG_KEY: "{'id': 'posthog_key', 'host': 'https://product.impress.127.0.0.1.nip.io'}"
DB_HOST: postgresql
DB_NAME: impress
DB_USER: dinum
DB_PASSWORD: pass
DB_PORT: 5432
POSTGRES_DB: impress
POSTGRES_USER: dinum
POSTGRES_PASSWORD: pass
REDIS_URL: redis://default:pass@redis-master:6379/1
AWS_S3_ENDPOINT_URL: http://minio.impress.svc.cluster.local:9000
AWS_S3_ACCESS_KEY_ID: root
AWS_S3_SECRET_ACCESS_KEY: password
AWS_STORAGE_BUCKET_NAME: impress-media-storage
STORAGES_STATICFILES_BACKEND: django.contrib.staticfiles.storage.StaticFilesStorage
Y_PROVIDER_API_BASE_URL: http://impress-y-provider:443/api/
Y_PROVIDER_API_KEY: my-secret
migrate:
command:
- "/bin/sh"
- "-c"
- |
python manage.py migrate --no-input &&
python manage.py create_demo --force
restartPolicy: Never
command:
- "gunicorn"
- "-c"
- "/usr/local/etc/gunicorn/impress.py"
- "impress.wsgi:application"
- "--reload"
createsuperuser:
command:
- "/bin/sh"
- "-c"
- |
python manage.py createsuperuser --email admin@example.com --password admin
restartPolicy: Never
# Extra volume to manage our local custom CA and avoid to set ssl_verify: false
extraVolumeMounts:
- name: certs
mountPath: /usr/local/lib/python3.13/site-packages/certifi/cacert.pem
subPath: cacert.pem
# Extra volume to manage our local custom CA and avoid to set ssl_verify: false
extraVolumes:
- name: certs
configMap:
name: certifi
items:
- key: cacert.pem
path: cacert.pem
frontend:
envVars:
PORT: 8080
NEXT_PUBLIC_API_ORIGIN: https://impress.127.0.0.1.nip.io
replicas: 1
image:
repository: lasuite/impress-frontend
pullPolicy: Always
tag: "latest"
yProvider:
replicas: 1
image:
repository: lasuite/impress-y-provider
pullPolicy: Always
tag: "latest"
envVars:
COLLABORATION_LOGGING: true
COLLABORATION_SERVER_ORIGIN: https://impress.127.0.0.1.nip.io
COLLABORATION_SERVER_SECRET: my-secret
Y_PROVIDER_API_KEY: my-secret
COLLABORATION_BACKEND_BASE_URL: https://impress.127.0.0.1.nip.io
NODE_EXTRA_CA_CERTS: /usr/local/share/ca-certificates/cacert.pem
# Mount the certificate so yProvider can establish tls with the backend
extraVolumeMounts:
- name: certs
mountPath: /usr/local/share/ca-certificates/cacert.pem
subPath: cacert.pem
extraVolumes:
- name: certs
configMap:
name: certifi
items:
- key: cacert.pem
path: cacert.pem
posthog:
ingress:
enabled: false
ingressAssets:
enabled: false
ingress:
enabled: true
host: impress.127.0.0.1.nip.io
ingressCollaborationWS:
enabled: true
host: impress.127.0.0.1.nip.io
ingressCollaborationApi:
enabled: true
host: impress.127.0.0.1.nip.io
ingressAdmin:
enabled: true
host: impress.127.0.0.1.nip.io
ingressMedia:
enabled: true
host: impress.127.0.0.1.nip.io
annotations:
nginx.ingress.kubernetes.io/auth-url: https://impress.127.0.0.1.nip.io/api/v1.0/documents/media-auth/
nginx.ingress.kubernetes.io/auth-response-headers: "Authorization, X-Amz-Date, X-Amz-Content-SHA256"
nginx.ingress.kubernetes.io/upstream-vhost: minio.impress.svc.cluster.local:9000
nginx.ingress.kubernetes.io/rewrite-target: /impress-media-storage/$1
serviceMedia:
host: minio.impress.svc.cluster.local
port: 9000

File diff suppressed because it is too large Load Diff

View File

@@ -1,8 +0,0 @@
auth:
rootUser: root
rootPassword: password
provisioning:
enabled: true
buckets:
- name: impress-media-storage
versioning: true

View File

@@ -1,7 +0,0 @@
auth:
username: dinum
password: pass
database: impress
tls:
enabled: true
autoGenerated: true

View File

@@ -1,4 +0,0 @@
auth:
password: pass
architecture: standalone

View File

@@ -1,226 +0,0 @@
# Installation with docker compose
We provide a sample configuration for running Docs using Docker Compose. Please note that this configuration is experimental, and the official way to deploy Docs in production is to use [k8s](../installation/k8s.md)
## Requirements
- A modern version of Docker and its Compose plugin.
- A domain name and DNS configured to your server.
- An Identity Provider that supports OpenID Connect protocol - we provide [an example to deploy Keycloak](../examples/compose/keycloak/README.md).
- An Object Storage that implements S3 API - we provide [an example to deploy Minio](../examples/compose/minio/README.md).
- A Postgresql database - we provide [an example in the compose file](../examples/compose/compose.yaml).
- A Redis database - we provide [an example in the compose file](../examples/compose/compose.yaml).
## Software Requirements
Ensure you have Docker Compose(v2) installed on your host server. Follow the official guidelines for a reliable setup:
Docker Compose is included with Docker Engine:
- **Docker Engine:** We suggest adhering to the instructions provided by Docker
for [installing Docker Engine](https://docs.docker.com/engine/install/).
For older versions of Docker Engine that do not include Docker Compose:
- **Docker Compose:** Install it as per the [official documentation](https://docs.docker.com/compose/install/).
> [!NOTE]
> `docker-compose` may not be supported. You are advised to use `docker compose` instead.
## Step 1: Prepare your working environment:
```bash
mkdir -p docs/env.d
curl -o compose.yaml https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/docs/examples/compose/compose.yaml
curl -o env.d/common https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/env.d/production.dist/common
curl -o env.d/backend https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/env.d/production.dist/backend
curl -o env.d/yprovider https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/env.d/production.dist/yprovider
curl -o env.d/common https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/env.d/production.dist/postgresql
```
## Step 2: Configuration
Docs configuration is achieved through environment variables. We provide a [detailed description of all variables](../env.md).
In this example, we assume the following services:
- OIDC provider on https://id.yourdomain.tld
- Object Storage on https://storage.yourdomain.tld
- Docs on https://docs.yourdomain.tld
- Bucket name is docs-media-storage
**Set your own values in `env.d/common`**
### OIDC
Authentication in Docs is managed through Open ID Connect protocol. A functional Identity Provider implementing this protocol is required.
For guidance, refer to our [Keycloak deployment example](../examples/compose/keycloak/README.md).
If using Keycloak as your Identity Provider, set `OIDC_RP_CLIENT_ID` and `OIDC_RP_CLIENT_SECRET` variables with those of the OIDC client created for Docs. By default we have set `docs` as the realm name, if you have named your realm differently, update the value `REALM_NAME` in `env.d/common`
For others OIDC providers, update the variables in `env.d/backend`.
### Object Storage
Files and media are stored in an Object Store that supports the S3 API.
For guidance, refer to our [Minio deployment example](../examples/compose/minio/README.md).
Set `AWS_S3_ACCESS_KEY_ID` and `AWS_S3_SECRET_ACCESS_KEY` with the credentials of a user with `readwrite` access to the bucket created for Docs.
### Postgresql
Docs uses PostgreSQL as its database. Although an external PostgreSQL can be used, our example provides a deployment method.
If you are using the example provided, you need to generate a secure key for `DB_PASSWORD` and set it in `env.d/postgresql`.
If you are using an external service or not using our default values, you should update the variables in `env.d/postgresql`
### Redis
Docs uses Redis for caching. While an external Redis can be used, our example provides a deployment method.
If you are using an external service, you need to set `REDIS_URL` environment variable in `env.d/backend`.
### Y Provider
The Y provider service enables collaboration through websockets.
Generates a secure key for `Y_PROVIDER_API_KEY` and `COLLABORATION_SERVER_SECRET` in ``env.d/yprovider``.
### Docs
The Docs backend is built on the Django Framework.
Generates a secure key for `DJANGO_SECRET_KEY` in `env.d/backend`.
### Logging
Update the following variables in `env.d/backend` if you want to change the logging levels:
```env
LOGGING_LEVEL_HANDLERS_CONSOLE=DEBUG
LOGGING_LEVEL_LOGGERS_ROOT=DEBUG
LOGGING_LEVEL_LOGGERS_APP=DEBUG
```
### Mail
The following environment variables are required in `env.d/backend` for the mail service to send invitations :
```env
DJANGO_EMAIL_HOST=<smtp host>
DJANGO_EMAIL_HOST_USER=<smtp user>
DJANGO_EMAIL_HOST_PASSWORD=<smtp password>
DJANGO_EMAIL_PORT=<smtp port>
DJANGO_EMAIL_FROM=<your email address>
#DJANGO_EMAIL_USE_TLS=true # A flag to enable or disable TLS for email sending.
#DJANGO_EMAIL_USE_SSL=true # A flag to enable or disable SSL for email sending.
DJANGO_EMAIL_BRAND_NAME=<brand name used in email templates> # e.g. "La Suite Numérique"
DJANGO_EMAIL_LOGO_IMG=<logo image to use in email templates.> # e.g. "https://docs.yourdomain.tld/assets/logo-suite-numerique.png"
```
### AI
Built-in AI actions let users generate, summarize, translate, and correct content.
AI is disabled by default. To enable it, the following environment variables must be set in in `env.d/backend`:
```env
AI_FEATURE_ENABLED=true # is false by default
AI_BASE_URL=https://openaiendpoint.com
AI_API_KEY=<API key>
AI_MODEL=<model used> e.g. llama
```
### Frontend theme
You can [customize your Docs instance](../theming.md) with your own theme and custom css.
The following environment variables must be set in `env.d/backend`:
```env
FRONTEND_THEME=default # name of your theme built with cuningham
FRONTEND_CSS_URL=https://storage.yourdomain.tld/themes/custom.css # custom css
```
## Step 3: Reverse proxy and SSL/TLS
> [!WARNING]
> In a production environment, configure SSL/TLS termination to run your instance on https.
If you have your own certificates and proxy setup, you can skip this part.
You can follow our [nginx proxy example](../examples/compose/nginx-proxy/README.md) with automatic generation and renewal of certificate with Let's Encrypt.
You will need to uncomment the environment and network sections in compose file and update it with your values.
```yaml
frontend:
...
# Uncomment and set your values if using our nginx proxy example
#environment:
# - VIRTUAL_HOST=${DOCS_HOST} # used by nginx proxy
# - VIRTUAL_PORT=8083 # used by nginx proxy
# - LETSENCRYPT_HOST=${DOCS_HOST} # used by lets encrypt to generate TLS certificate
...
# Uncomment if using our nginx proxy example
# networks:
# - proxy-tier
#
#networks:
# proxy-tier:
# external: true
```
## Step 4: Start Docs
You are ready to start your Docs application !
```bash
docker compose up -d
```
> [!NOTE]
> Version of the images are set to latest, you should pin it to the desired version to avoid unwanted upgrades when pulling latest image.
## Step 5: Run the database migration and create Django admin user
```bash
docker compose run --rm backend python manage.py migrate
docker compose run --rm backend python manage.py createsuperuser --email <admin email> --password <admin password>
```
Replace `<admin email>` with the email of your admin user and generate a secure password.
Your docs instance is now available on the domain you defined, https://docs.yourdomain.tld.
THe admin interface is available on https://docs.yourdomain.tld/admin with the admin user you just created.
## How to upgrade your Docs application
Before running an upgrade you must check the [Upgrade document](../../UPGRADE.md) for specific procedures that might be needed.
You can also check the [Changelog](../../CHANGELOG.md) for brief summary of the changes.
### Step 1: Edit the images tag with the desired version
### Step 2: Pull the images
```bash
docker compose pull
```
### Step 3: Restart your containers
```bash
docker compose restart
```
### Step 4: Run the database migration
Your database schema may need to be updated, run:
```bash
docker compose run --rm backend python manage.py migrate
```

View File

@@ -1,230 +0,0 @@
# Installation on a k8s cluster
This document is a step-by-step guide that describes how to install Docs on a k8s cluster without AI features. It's a teaching document to learn how it works. It needs to be adapted for a production environment.
## Prerequisites
- k8s cluster with an nginx-ingress controller
- an OIDC provider (if you don't have one, we provide an example)
- a PostgreSQL server (if you don't have one, we provide an example)
- a Memcached server (if you don't have one, we provide an example)
- a S3 bucket (if you don't have one, we provide an example)
### Test cluster
If you do not have a test cluster, you can install everything on a local Kind cluster. In this case, the simplest way is to use our script **bin/start-kind.sh**.
To be able to use the script, you need to install:
- Docker (https://docs.docker.com/desktop/)
- Kind (https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
- Mkcert (https://github.com/FiloSottile/mkcert#installation)
- Helm (https://helm.sh/docs/intro/quickstart/#install-helm)
```
./bin/start-kind.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 4700 100 4700 0 0 92867 0 --:--:-- --:--:-- --:--:-- 94000
0. Create ca
The local CA is already installed in the system trust store! 👍
The local CA is already installed in the Firefox and/or Chrome/Chromium trust store! 👍
Created a new certificate valid for the following names 📜
- "127.0.0.1.nip.io"
- "*.127.0.0.1.nip.io"
Reminder: X.509 wildcards only go one level deep, so this won't match a.b.127.0.0.1.nip.io
The certificate is at "./127.0.0.1.nip.io+1.pem" and the key at "./127.0.0.1.nip.io+1-key.pem" ✅
It will expire on 24 March 2027 🗓
1. Create registry container unless it already exists
2. Create kind cluster with containerd registry config dir enabled
Creating cluster "suite" ...
✓ Ensuring node image (kindest/node:v1.27.3) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-suite"
You can now use your cluster with:
kubectl cluster-info --context kind-suite
Thanks for using kind! 😊
3. Add the registry config to the nodes
4. Connect the registry to the cluster network if not already connected
5. Document the local registry
configmap/local-registry-hosting created
Warning: resource configmaps/coredns is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
configmap/coredns configured
deployment.apps/coredns restarted
6. Install ingress-nginx
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
secret/mkcert created
deployment.apps/ingress-nginx-controller patched
7. Setup namespace
namespace/impress created
Context "kind-suite" modified.
secret/mkcert created
$ kubectl -n ingress-nginx get po
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-t55ph 0/1 Completed 0 2m56s
ingress-nginx-admission-patch-94dvt 0/1 Completed 1 2m56s
ingress-nginx-controller-57c548c4cd-2rx47 1/1 Running 0 2m56s
```
When your k8s cluster is ready (the ingress nginx controller is up), you can start the deployment. This cluster is special because it uses the `*.127.0.0.1.nip.io` domain and mkcert certificates to have full HTTPS support and easy domain name management.
Please remember that `*.127.0.0.1.nip.io` will always resolve to `127.0.0.1`, except in the k8s cluster where we configure CoreDNS to answer with the ingress-nginx service IP.
## Preparation
### What do you use to authenticate your users?
Docs uses OIDC, so if you already have an OIDC provider, obtain the necessary information to use it. In the next step, we will see how to configure Django (and thus Docs) to use it. If you do not have a provider, we will show you how to deploy a local Keycloak instance (this is not a production deployment, just a demo).
```
$ kubectl create namespace impress
$ kubectl config set-context --current --namespace=impress
$ helm install keycloak oci://registry-1.docker.io/bitnamicharts/keycloak -f examples/keycloak.values.yaml
$ #wait until
$ kubectl get po
NAME READY STATUS RESTARTS AGE
keycloak-0 1/1 Running 0 6m48s
keycloak-postgresql-0 1/1 Running 0 6m48s
```
From here the important information you will need are:
```yaml
OIDC_OP_JWKS_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/certs
OIDC_OP_AUTHORIZATION_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/token
OIDC_OP_USER_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/userinfo
OIDC_OP_LOGOUT_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/impress/protocol/openid-connect/session/end
OIDC_RP_CLIENT_ID: impress
OIDC_RP_CLIENT_SECRET: ThisIsAnExampleKeyForDevPurposeOnly
OIDC_RP_SIGN_ALGO: RS256
OIDC_RP_SCOPES: "openid email"
```
You can find these values in **examples/keycloak.values.yaml**
### Find redis server connection values
Docs needs a redis so we start by deploying one:
```
$ helm install redis oci://registry-1.docker.io/bitnamicharts/redis -f examples/redis.values.yaml
$ kubectl get po
NAME READY STATUS RESTARTS AGE
keycloak-0 1/1 Running 0 26m
keycloak-postgresql-0 1/1 Running 0 26m
redis-master-0 1/1 Running 0 35s
```
### Find postgresql connection values
Docs uses a postgresql database as backend, so if you have a provider, obtain the necessary information to use it. If you don't, you can install a postgresql testing environment as follow:
```
$ helm install postgresql oci://registry-1.docker.io/bitnamicharts/postgresql -f examples/postgresql.values.yaml
$ kubectl get po
NAME READY STATUS RESTARTS AGE
keycloak-0 1/1 Running 0 28m
keycloak-postgresql-0 1/1 Running 0 28m
postgresql-0 1/1 Running 0 14m
redis-master-0 1/1 Running 0 42s
```
From here the important information you will need are:
```yaml
DB_HOST: postgres-postgresql
DB_NAME: impress
DB_USER: dinum
DB_PASSWORD: pass
DB_PORT: 5432
POSTGRES_DB: impress
POSTGRES_USER: dinum
POSTGRES_PASSWORD: pass
```
### Find s3 bucket connection values
Docs uses an s3 bucket to store documents, so if you have a provider obtain the necessary information to use it. If you don't, you can install a local minio testing environment as follow:
```
$ helm install minio oci://registry-1.docker.io/bitnamicharts/minio -f examples/minio.values.yaml
$ kubectl get po
NAME READY STATUS RESTARTS AGE
keycloak-0 1/1 Running 0 38m
keycloak-postgresql-0 1/1 Running 0 38m
minio-84f5c66895-bbhsk 1/1 Running 0 42s
minio-provisioning-2b5sq 0/1 Completed 0 42s
postgresql-0 1/1 Running 0 24m
redis-master-0 1/1 Running 0 10m
```
## Deployment
Now you are ready to deploy Docs without AI. AI requires more dependencies (OpenAI API). To deploy Docs you need to provide all previous information to the helm chart.
```
$ helm repo add impress https://suitenumerique.github.io/docs/
$ helm repo update
$ helm install impress impress/docs -f examples/impress.values.yaml
$ kubectl get po
NAME READY STATUS RESTARTS AGE
impress-docs-backend-96558758d-xtkbp 0/1 Running 0 79s
impress-docs-backend-createsuperuser-r7ltc 0/1 Completed 0 79s
impress-docs-backend-migrate-c949s 0/1 Completed 0 79s
impress-docs-frontend-6749f644f7-p5s42 1/1 Running 0 79s
impress-docs-y-provider-6947fd8f54-78f2l 1/1 Running 0 79s
keycloak-0 1/1 Running 0 48m
keycloak-postgresql-0 1/1 Running 0 48m
minio-84f5c66895-bbhsk 1/1 Running 0 10m
minio-provisioning-2b5sq 0/1 Completed 0 10m
postgresql-0 1/1 Running 0 34m
redis-master-0 1/1 Running 0 20m
```
## Test your deployment
In order to test your deployment you have to log into your instance. If you exclusively use our examples you can do:
```
$ kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
impress-docs <none> impress.127.0.0.1.nip.io localhost 80, 443 114s
impress-docs-admin <none> impress.127.0.0.1.nip.io localhost 80, 443 114s
impress-docs-collaboration-api <none> impress.127.0.0.1.nip.io localhost 80, 443 114s
impress-docs-media <none> impress.127.0.0.1.nip.io localhost 80, 443 114s
impress-docs-ws <none> impress.127.0.0.1.nip.io localhost 80, 443 114s
keycloak <none> keycloak.127.0.0.1.nip.io localhost 80 49m
```
You can use Docs at https://impress.127.0.0.1.nip.io. The provisionning user in keycloak is impress/impress.

235
docs/models.md Normal file
View File

@@ -0,0 +1,235 @@
# What is People?
Space Odyssey is a dynamic organization. They use the People application to enhance teamwork and
streamline communication among their co-workers. Let's explore how this application helps them
interact efficiently.
Let's see how we could interact with Django's shell to recreate David's environment in the app.
## Base contacts from the organization records
David Bowman is an exemplary employee at Space Odyssey Corporation. His email is
`david.bowman@spaceodyssey.com` and he is registered in the organization's records via a base
contact as follows:
```python
david_base_contact = Contact.objects.create(
full_name="David Bowman",
short_name="David",
data={
"emails": [
{"type": "Work", "value": "david.bowman@spaceodyssey.com"},
],
"phones": [
{"type": "Work", "value": "(123) 456-7890"},
],
"addresses": [
{
"type": "Work",
"street": "123 Main St",
"city": "Cityville",
"state": "CA",
"zip": "12345",
"country": "USA",
}
],
"links": [
{"type": "Website", "value": "http://www.spaceodyssey.com"},
{"type": "Twitter", "value": "https://www.twitter.com/dbowman"},
],
"organizations": [
{
"name": "Space Odyssey Corporation",
"department": "IT",
"jobTitle": "AI Engineer",
},
],
}
)
```
When David logs-in to the People application for the first time using the corporation's OIDC
Single Sign-On service. A user is created for him on the fly by the system, together with an
identity record representing the OIDC session:
```python
david_user = User.objects.create(
language="en-us",
timezone="America/Los_Angeles",
)
david_identity = Identity.objects.create(
"user": david_user,
"sub": "2a1b3c4d-5e6f-7a8b-9c0d-1e2f3a4b5c6d",
"email" : "david.bowman@spaceodyssey.com",
"is_main": True,
)
```
## Profile contact
The system identifies Dave through the email associated with his OIDC session and prompts him to
confirm the details of the base contact stored in the database.
When David confirms, giving an alternative short name that he prefers, a contact override is
created on top of the organization's base contact. This new contact is marked as David's profile
on the user:
```python
david_contact = Contact.objects.create(
base=david_base_contact,
owner=david_user,
full_name="David Bowman",
short_name="Dave",
data={}
)
david_user.profile_contact = david_contact
david_user.save()
```
If Dave had not had any existing contact in the organization's records, the profile contact would
have been created independently, without any connection to a base contact:
```python
david_contact = Contact.objects.create(
base=None,
owner=david_user,
full_name="David Bowman",
short_name="Dave",
data={}
)
```
Now, Dave feels like sharing his mobile phone number with his colleagues. He can do this
by editing his contact in the application:
```python
contact.data["phones"] = [
{"type": "Mobile", "value": "(123) 456-7890"},
]
contact.save()
```
## Contact override
During a Space conference he attended, Dave met Dr Ryan Stone, a medical engineer who gave him
her professional email address. Ryan is already present in the system but her email is missing.
Dave can add it to his private version of the contact:
```python
ryan_base_contact = Contact.objects.create(
full_name="Ryan Stone",
data={}
)
ryan_contact = Contact.objects.create(
base=ryan_base_contact,
owner=david_user,
full_name="Ryan Stone",
short_name="Dr Ryan",
data={
"emails": [
{"type": "Work", "value": "ryan.stone@hubblestation.com"},
],
}
)
```
## Team Collaboration
Dave wants to form a team with Ryan and other colleagues to work together better on using the organization's digital tools for their projects.
Dave would like to create a team with Ryan and some other colleagues, to enhance collaboration
throughout their projects:
```python
projectx = Team.objects.create(name="Project X")
```
A team can for example be used to create an email alias or to define role based access rights
(RBAC) in a specific application or all applications of the organization's digital Suite.
Having created he team, Dave is automatically assigned the "owner" role. He invites Ryan,
granting an "administrator" role to her so she can invite her own colleagues. Both of them can
then proceed to invite other colleagues as simple members. If Ryan wants, she can upgrade a
colleague to "administrator" but only David can upgrade someone to the "owner" status:
```python
TeamAccess.objects.create(user=david_user, team=projectx, role="owner")
TeamAccess.objects.create(user=ryan_user, team=projectx, role="administrator")
TeamAccess.objects.create(user=julie_user, team=projectx, role="member")
```
| Role | Member | Administrator | Owner |
|-----------------------------------|--------|---------------|-------|
| Can view team | ✔ | ✔ | ✔ |
| Can set roles except for owners | | ✔ | ✔ |
| Can set roles for owners | | | ✔ |
| Can delete team | | | ✔ |
Importantly, the system ensures that there is always at least one owner left to maintain control
of the team.
# Models overview
The following graph represents the application's models and their relationships:
```mermaid
erDiagram
%% Models
Contact {
UUID id PK
Contact base
User owner
string full_name
string short_name
json data
DateTime created_at
DateTime updated_at
}
User {
UUID id PK
Contact profile_contact
string language
string timezone
boolean is_device
boolean is_staff
boolean is_active
DateTime created_at
DateTime updated_at
}
Identity {
UUID id PK
User user
string sub
Email email
boolean is_main
DateTime created_at
DateTime updated_at
}
Team {
UUID id PK
string name
DateTime created_at
DateTime updated_at
}
TeamAccess {
UUID id PK
Team team
User user
string role
DateTime created_at
DateTime updated_at
}
%% Relations
User ||--o{ Contact : "owns"
Contact ||--o{ User : "profile for"
User ||--o{ TeamAccess : ""
Team ||--o{ TeamAccess : ""
Identity ||--o{ User : "connects"
Contact }o--|| Contact : "overrides"
```

View File

@@ -1,72 +0,0 @@
# Releasing a new version
Whenever we are cooking a new release (e.g. `4.18.1`) we should follow a standard procedure described below:
1. Create a new branch named: `release/4.18.1`.
2. Bump the release number for backend project, frontend projects, and Helm files:
- for backend, update the version number by hand in `pyproject.toml`,
- for each projects (`src/frontend`, `src/frontend/apps/*`, `src/frontend/packages/*`, `src/mail`), run `yarn version --new-version --no-git-tag-version 4.18.1` in their directory. This will update their `package.json` for you,
- for Helm, update Docker image tag in files located at `src/helm/env.d` for both `preprod` and `production` environments:
```yaml
image:
repository: lasuite/impress-backend
pullPolicy: Always
tag: "v4.18.1" # Replace with your new version number, without forgetting the "v" prefix
...
frontend:
image:
repository: lasuite/impress-frontend
pullPolicy: Always
tag: "v4.18.1"
y-provider:
image:
repository: lasuite/impress-y-provider
pullPolicy: Always
tag: "v4.18.1"
```
The new images don't exist _yet_: they will be created automatically later in the process.
3. Update the project's `Changelog` following the [keepachangelog](https://keepachangelog.com/en/0.3.0/) recommendations
4. Commit your changes with the following format: the 🔖 release emoji, the type of release (patch/minor/patch) and the release version:
```text
🔖(minor) bump release to 4.18.0
```
5. Open a pull request, wait for an approval from your peers and merge it.
6. Checkout and pull changes from the `main` branch to ensure you have the latest updates.
7. Tag and push your commit:
```bash
git tag v4.18.1 && git push origin tag v4.18.1
```
Doing this triggers the CI and tells it to build the new Docker image versions that you targeted earlier in the Helm files.
8. Ensure the new [backend](https://hub.docker.com/r/lasuite/impress-frontend/tags) and [frontend](https://hub.docker.com/r/lasuite/impress-frontend/tags) image tags are on Docker Hub.
9. The release is now done!
# Deploying
> [!TIP]
> The `staging` platform is deployed automatically with every update of the `main` branch.
Making a new release doesn't publish it automatically in production.
Deployment is done by ArgoCD. ArgoCD checks for the `production` tag and automatically deploys the production platform with the targeted commit.
To publish, we mark the commit we want with the `production` tag. ArgoCD is then notified that the tag has changed. It then deploys the Docker image tags specified in the Helm files of the targeted commit.
To publish the release you just made:
```bash
git tag --force production v4.18.1
git push --force origin production
```

View File

@@ -1,110 +0,0 @@
# La Suite Docs System & Requirements (2025-06)
## 1. Quick-Reference Matrix (single VM / laptop)
| Scenario | RAM | vCPU | SSD | Notes |
| ------------------------- | ----- | ---- | ------- | ------------------------- |
| **Solo dev** | 8 GB | 4 | 15 GB | Hot-reload + one IDE |
| **Team QA** | 16 GB | 6 | 30 GB | Runs integration tests |
| **Prod ≤ 100 live users** | 32 GB | 8 + | 50 GB + | Scale linearly above this |
Memory is the first bottleneck; CPU matters only when Celery or the Next.js build is saturated.
> **Note:** Memory consumption varies by operating system. Windows tends to be more memory-hungry than Linux, so consider adding 10-20% extra RAM when running on Windows compared to Linux-based systems.
## 2. Development Environment Memory Requirements
| Service | Typical use | Rationale / source |
| ------------------------ | ----------------------------- | --------------------------------------------------------------------------------------- |
| PostgreSQL | **1 2 GB** | `shared_buffers` starting point ≈ 25% RAM ([postgresql.org][1]) |
| Keycloak | **≈ 1.3 GB** | 70% of limit for heap + ~300 MB non-heap ([keycloak.org][2]) |
| Redis | **≤ 256 MB** | Empty instance ≈ 3 MB; budget 256 MB to allow small datasets ([stackoverflow.com][3]) |
| MinIO | **2 GB (dev) / 32 GB (prod)**| Pre-allocates 12 GiB; docs recommend 32 GB per host for ≤ 100 Ti storage ([min.io][4]) |
| Django API (+ Celery) | **0.8 1.5 GB** | Empirical in-house metrics |
| Next.js frontend | **0.5 1 GB** | Dev build chain |
| Y-Provider (y-websocket) | **< 200 MB** | Large 40 MB YDoc called “big” in community thread ([discuss.yjs.dev][5]) |
| Nginx | **< 100 MB** | Static reverse-proxy footprint |
[1]: https://www.postgresql.org/docs/9.1/runtime-config-resource.html "PostgreSQL: Documentation: 9.1: Resource Consumption"
[2]: https://www.keycloak.org/high-availability/concepts-memory-and-cpu-sizing "Concepts for sizing CPU and memory resources - Keycloak"
[3]: https://stackoverflow.com/questions/45233052/memory-footprint-for-redis-empty-instance "Memory footprint for Redis empty instance - Stack Overflow"
[4]: https://min.io/docs/minio/kubernetes/upstream/operations/checklists/hardware.html "Hardware Checklist — MinIO Object Storage for Kubernetes"
[5]: https://discuss.yjs.dev/t/understanding-memory-requirements-for-production-usage/198 "Understanding memory requirements for production usage - Yjs Community"
> **Rule of thumb:** add 2 GB for OS/overhead, then sum only the rows you actually run.
## 3. Production Environment Memory Requirements
Production deployments differ significantly from development environments. The table below shows typical memory usage for production services:
| Service | Typical use | Rationale / notes |
| ------------------------ | ----------------------------- | --------------------------------------------------------------------------------------- |
| PostgreSQL | **2 8 GB** | Higher `shared_buffers` and connection pooling for concurrent users |
| OIDC Provider (optional) | **Variable** | Any OIDC-compatible provider (Keycloak, Auth0, Azure AD, etc.) - external or self-hosted |
| Redis | **256 MB 2 GB** | Session storage and caching; scales with active user sessions |
| Object Storage (optional)| **External or self-hosted** | Can use AWS S3, Azure Blob, Google Cloud Storage, or self-hosted MinIO |
| Django API (+ Celery) | **1 3 GB** | Production workloads with background tasks and higher concurrency |
| Static Files (Nginx) | **< 200 MB** | Serves Next.js build output and static assets; no development overhead |
| Y-Provider (y-websocket) | **200 MB 1 GB** | Scales with concurrent document editing sessions |
| Nginx (Load Balancer) | **< 200 MB** | Reverse proxy, SSL termination, static file serving |
### Production Architecture Notes
- **Frontend**: Uses pre-built Next.js static assets served by Nginx (no Node.js runtime needed)
- **Authentication**: Any OIDC-compatible provider can be used instead of self-hosted Keycloak
- **Object Storage**: External services (S3, Azure Blob) or self-hosted solutions (MinIO) are both viable
- **Database**: Consider PostgreSQL clustering or managed database services for high availability
- **Scaling**: Horizontal scaling is recommended for Django API and Y-Provider services
### Minimal Production Setup (Core Services Only)
| Service | Memory | Notes |
| ------------------------ | --------- | --------------------------------------- |
| PostgreSQL | **2 GB** | Core database |
| Django API (+ Celery) | **1.5 GB**| Backend services |
| Y-Provider | **200 MB**| Real-time collaboration |
| Nginx | **100 MB**| Static files + reverse proxy |
| Redis | **256 MB**| Session storage |
| **Total (without auth/storage)** | **≈ 4 GB** | External OIDC + object storage assumed |
## 4. Recommended Software Versions
| Tool | Minimum |
| ----------------------- | ------- |
| Docker Engine / Desktop | 24.0 |
| Docker Compose | v2 |
| Git | 2.40 |
| **Node.js** | 22+ |
| **Python** | 3.13+ |
| GNU Make | 4.4 |
| Kind | 0.22 |
| Helm | 3.14 |
| kubectl | 1.29 |
| mkcert | 1.4 |
## 5. Ports (dev defaults)
| Port | Service |
| --------- | --------------------- |
| 3000 | Next.js |
| 8071 | Django |
| 4444 | Y-Provider |
| 8080 | Keycloak |
| 8083 | Nginx proxy |
| 9000/9001 | MinIO |
| 15432 | PostgreSQL (main) |
| 5433 | PostgreSQL (Keycloak) |
| 1081 | MailCatcher |
## 6. Sizing Guidelines
**RAM** start at 8 GB dev / 16 GB staging / 32 GB prod. Postgres and Keycloak are the first to OOM; scale them first.
> **OS considerations:** Windows systems typically require 10-20% more RAM than Linux due to higher OS overhead. Docker Desktop on Windows also uses additional memory compared to native Linux Docker.
**CPU** budget one vCPU per busy container until Celery or Next.js builds saturate.
**Disk** SSD; add 10 GB extra for the Docker layer cache.
**MinIO** for demos, mount a local folder instead of running MinIO to save 2 GB+ of RAM.

View File

@@ -1,70 +0,0 @@
# Runtime Theming 🎨
### How to Use
To use this feature, simply set the `FRONTEND_CSS_URL` environment variable to the URL of your custom CSS file. For example:
```javascript
FRONTEND_CSS_URL=http://anything/custom-style.css
```
Once you've set this variable, our application will load your custom CSS file and apply the styles to our frontend application.
### Benefits
This feature provides several benefits, including:
* **Easy customization** 🔄: With this feature, you can easily customize the look and feel of our application without requiring any code changes.
* **Flexibility** 🌈: You can use any CSS styles you like to create a custom theme that meets your needs.
* **Runtime theming** ⏱️: This feature allows you to change the theme of our application at runtime, without requiring a restart or recompilation.
### Example Use Case
Let's say you want to change the background color of our application to a custom color. You can create a custom CSS file with the following contents:
```css
body {
background-color: #3498db;
}
```
Then, set the `FRONTEND_CSS_URL` environment variable to the URL of your custom CSS file. Once you've done this, our application will load your custom CSS file and apply the styles, changing the background color to the custom color you specified.
----
# **Footer Configuration** 📝
The footer is configurable from the theme customization file.
### Settings 🔧
```shellscript
THEME_CUSTOMIZATION_FILE_PATH=<path>
```
### Example of JSON
The json must follow some rules: https://github.com/suitenumerique/docs/blob/main/src/helm/env.d/dev/configuration/theme/demo.json
`footer.default` is the fallback if the language is not supported.
---
Below is a visual example of a configured footer ⬇️:
![Footer Configuration Example](./assets/footer-configurable.png)
----
# **Custom Translations** 📝
The translations can be partially overridden from the theme customization file.
### Settings 🔧
```shellscript
THEME_CUSTOMIZATION_FILE_PATH=<path>
```
### Example of JSON
The json must follow some rules: https://github.com/suitenumerique/docs/blob/main/src/helm/env.d/dev/configuration/theme/demo.json

View File

@@ -1,194 +0,0 @@
# Troubleshooting Guide
## Line Ending Issues on Windows (LF/CRLF)
### Problem Description
This project uses **LF (Line Feed: `\n`) line endings** exclusively. Windows users may encounter issues because:
- **Windows** defaults to CRLF (Carriage Return + Line Feed: `\r\n`) for line endings
- **This project** uses LF line endings for consistency across all platforms
- **Git** may automatically convert line endings, causing conflicts or build failures
### Common Symptoms
- Git shows files as modified even when no changes were made
- Error messages like "warning: LF will be replaced by CRLF"
- Build failures or linting errors due to line ending mismatches
### Solutions for Windows Users
#### Configure Git to Preserve LF (Recommended)
Configure Git to NOT convert line endings and preserve LF:
```bash
git config core.autocrlf false
git config core.eol lf
```
This tells Git to:
- Never convert line endings automatically
- Always use LF for line endings in working directory
#### Fix Existing Repository with Wrong Line Endings
If you already have CRLF line endings in your local repository, the **best approach** is to configure Git properly and clone the project again:
1. **Configure Git first**:
```bash
git config --global core.autocrlf false
git config --global core.eol lf
```
2. **Clone the project fresh** (recommended):
```bash
# Navigate to parent directory
cd ..
# Remove current repository (backup your changes first!)
rm -rf docs
# Clone again with correct line endings
git clone git@github.com:suitenumerique/docs.git
```
**Alternative**: If you have uncommitted changes and cannot re-clone:
1. **Backup your changes**:
```bash
git add .
git commit -m "Save changes before fixing line endings"
```
2. **Remove all files from Git's index**:
```bash
git rm --cached -r .
```
3. **Reset Git configuration** (if not done globally):
```bash
git config core.autocrlf false
git config core.eol lf
```
4. **Re-add all files** (Git will use LF line endings):
```bash
git add .
```
5. **Commit the changes**:
```bash
git commit -m "✏️(project) Fix line endings to LF"
```
## Minio Permission Issues on Windows
### Problem Description
On Windows, you may encounter permission-related errors when running Minio in development mode with Docker Compose. This typically happens because:
- **Windows file permissions** don't map well to Unix-style user IDs used in Docker containers
- **Docker Desktop** may have issues with user mapping when using the `DOCKER_USER` environment variable
- **Minio container** fails to start or access volumes due to permission conflicts
### Common Symptoms
- Minio container fails to start with permission denied errors
- Error messages related to file system permissions in Minio logs
- Unable to create or access buckets in the development environment
- Docker Compose showing Minio service as unhealthy or exited
### Solution for Windows Users
If you encounter Minio permission issues on Windows, you can temporarily disable user mapping for the Minio service:
1. **Open the `compose.yml` file**
2. **Comment out the user directive** in the `minio` service section:
```yaml
minio:
# user: ${DOCKER_USER:-1000} # Comment this line on Windows if permission issues occur
image: minio/minio
environment:
- MINIO_ROOT_USER=impress
- MINIO_ROOT_PASSWORD=password
# ... rest of the configuration
```
3. **Restart the services**:
```bash
make run
```
### Why This Works
- Commenting out the `user` directive allows the Minio container to run with its default user
- This bypasses Windows-specific permission mapping issues
- The container will have the necessary permissions to access and manage the mounted volumes
### Note
This is a **development-only workaround**. In production environments, proper user mapping and security considerations should be maintained according to your deployment requirements.
## Frontend File Watching Issues on Windows
### Problem Description
Windows users may experience issues with file watching in the frontend-development container. This typically happens because:
- **Docker on Windows** has known limitations with file change detection
- **Node.js file watchers** may not detect changes properly on Windows filesystem
- **Hot reloading** fails to trigger when files are modified
### Common Symptoms
- Changes to frontend code aren't detected automatically
- Hot module replacement doesn't work as expected
- Need to manually restart the frontend container after code changes
- Console shows no reaction when saving files
### Solution: Enable WATCHPACK_POLLING
Add the `WATCHPACK_POLLING=true` environment variable to the frontend-development service in your local environment:
1. **Modify the `compose.yml` file** by adding the environment variable to the frontend-development service:
```yaml
frontend-development:
user: "${DOCKER_USER:-1000}"
build:
context: .
dockerfile: ./src/frontend/Dockerfile
target: impress-dev
args:
API_ORIGIN: "http://localhost:8071"
PUBLISH_AS_MIT: "false"
SW_DEACTIVATED: "true"
image: impress:frontend-development
environment:
- WATCHPACK_POLLING=true # Add this line for Windows users
volumes:
- ./src/frontend:/home/frontend
- /home/frontend/node_modules
- /home/frontend/apps/impress/node_modules
ports:
- "3000:3000"
```
2. **Restart your containers**:
```bash
make run
```
### Why This Works
- `WATCHPACK_POLLING=true` forces the file watcher to use polling instead of filesystem events
- Polling periodically checks for file changes rather than relying on OS-level file events
- This is more reliable on Windows but slightly increases CPU usage
- Changes to your frontend code should now be detected properly, enabling hot reloading
### Note
This setting is primarily needed for Windows users. Linux and macOS users typically don't need this setting as file watching works correctly by default on those platforms.

25
docs/tsclient.md Normal file
View File

@@ -0,0 +1,25 @@
# Api client TypeScript
The backend application can automatically create a TypeScript client to be used in frontend
applications. It is used in the People front application itself.
This client is made with [openapi-typescript-codegen](https://github.com/ferdikoomen/openapi-typescript-codegen)
and People's backend OpenAPI schema (available [here](http://localhost:8071/v1.0/swagger/) if you have the backend running).
## Requirements
We'll need the online OpenAPI schema generated by swagger. Therefore you will first need to
install the backend application.
## Install openApiClientJs
```sh
$ cd src/tsclient
$ yarn install
```
## Generate the client
```sh
yarn generate:api:client:local <output_path_for_generated_client>
```

View File

@@ -1,43 +1,28 @@
# Django
DJANGO_ALLOWED_HOSTS=*
DJANGO_SECRET_KEY=ThisIsAnExampleKeyForDevPurposeOnly
DJANGO_SETTINGS_MODULE=impress.settings
DJANGO_SETTINGS_MODULE=people.settings
DJANGO_SUPERUSER_PASSWORD=admin
# Logging
# Set to DEBUG level for dev only
LOGGING_LEVEL_HANDLERS_CONSOLE=INFO
LOGGING_LEVEL_LOGGERS_ROOT=INFO
LOGGING_LEVEL_LOGGERS_APP=INFO
# Python
PYTHONPATH=/app
# impress settings
# People settings
# Mail
DJANGO_EMAIL_BRAND_NAME="La Suite Numérique"
DJANGO_EMAIL_HOST="mailcatcher"
DJANGO_EMAIL_LOGO_IMG="http://localhost:3000/assets/logo-suite-numerique.png"
DJANGO_EMAIL_PORT=1025
# Backend url
IMPRESS_BASE_URL="http://localhost:8072"
# Media
STORAGES_STATICFILES_BACKEND=django.contrib.staticfiles.storage.StaticFilesStorage
AWS_S3_ENDPOINT_URL=http://minio:9000
AWS_S3_ACCESS_KEY_ID=impress
AWS_S3_SECRET_ACCESS_KEY=password
MEDIA_BASE_URL=http://localhost:8083
PEOPLE_BASE_URL="http://localhost:8072"
# OIDC
OIDC_OP_JWKS_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/certs
OIDC_OP_AUTHORIZATION_ENDPOINT=http://localhost:8083/realms/impress/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/token
OIDC_OP_USER_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/userinfo
OIDC_OP_JWKS_ENDPOINT=http://nginx:8083/realms/people/protocol/openid-connect/certs
OIDC_OP_AUTHORIZATION_ENDPOINT=http://localhost:8083/realms/people/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT=http://nginx:8083/realms/people/protocol/openid-connect/token
OIDC_OP_USER_ENDPOINT=http://nginx:8083/realms/people/protocol/openid-connect/userinfo
OIDC_RP_CLIENT_ID=impress
OIDC_RP_CLIENT_ID=people
OIDC_RP_CLIENT_SECRET=ThisIsAnExampleKeyForDevPurposeOnly
OIDC_RP_SIGN_ALGO=RS256
OIDC_RP_SCOPES="openid email"
@@ -48,20 +33,3 @@ LOGOUT_REDIRECT_URL=http://localhost:3000
OIDC_REDIRECT_ALLOWED_HOSTS=["http://localhost:8083", "http://localhost:3000"]
OIDC_AUTH_REQUEST_EXTRA_PARAMS={"acr_values": "eidas1"}
# AI
AI_FEATURE_ENABLED=true
AI_BASE_URL=https://openaiendpoint.com
AI_API_KEY=password
AI_MODEL=llama
# Collaboration
COLLABORATION_API_URL=http://y-provider-development:4444/collaboration/api/
COLLABORATION_BACKEND_BASE_URL=http://app-dev:8000
COLLABORATION_SERVER_ORIGIN=http://localhost:3000
COLLABORATION_SERVER_SECRET=my-secret
COLLABORATION_WS_URL=ws://localhost:4444/collaboration/ws/
DJANGO_SERVER_TO_SERVER_API_TOKENS=server-api-token
Y_PROVIDER_API_BASE_URL=http://y-provider-development:4444/api/
Y_PROVIDER_API_KEY=yprovider-api-key

View File

@@ -1,9 +1,3 @@
# For the CI job test-e2e
BURST_THROTTLE_RATES="200/minute"
COLLABORATION_API_URL=http://y-provider:4444/collaboration/api/
SUSTAINED_THROTTLE_RATES="200/hour"
Y_PROVIDER_API_BASE_URL=http://y-provider:4444/api/
THEME_CUSTOMIZATION_FILE_PATH="" #force theme_customization to be empty
#COLLABORATION_API_URL=http://y-provider:4444/collaboration/api/
#Y_PROVIDER_API_BASE_URL=http://y-provider:4444/api/
BURST_THROTTLE_RATES="200/minute"

View File

@@ -1,3 +1,3 @@
CROWDIN_PERSONAL_TOKEN=Your-Personal-Token
CROWDIN_API_TOKEN=Your-Api-Token
CROWDIN_PROJECT_ID=Your-Project-Id
CROWDIN_BASE_PATH=/app/src

View File

@@ -1,11 +1,11 @@
# Postgresql db container configuration
POSTGRES_DB=keycloak
POSTGRES_USER=impress
POSTGRES_USER=people
POSTGRES_PASSWORD=pass
# App database configuration
DB_HOST=kc_postgresql
DB_NAME=keycloak
DB_USER=impress
DB_USER=people
DB_PASSWORD=pass
DB_PORT=5433
DB_PORT=5433

View File

@@ -1,11 +1,11 @@
# Postgresql db container configuration
POSTGRES_DB=impress
POSTGRES_DB=people
POSTGRES_USER=dinum
POSTGRES_PASSWORD=pass
# App database configuration
DB_HOST=postgresql
DB_NAME=impress
DB_NAME=people
DB_USER=dinum
DB_PASSWORD=pass
DB_PORT=5432
DB_PORT=5432

View File

@@ -1,65 +0,0 @@
## Django
DJANGO_ALLOWED_HOSTS=${DOCS_HOST}
DJANGO_SECRET_KEY=<generate a random key>
DJANGO_SETTINGS_MODULE=impress.settings
DJANGO_CONFIGURATION=Production
# Logging
# Set to DEBUG level for dev only
LOGGING_LEVEL_HANDLERS_CONSOLE=ERROR
LOGGING_LEVEL_LOGGERS_ROOT=INFO
LOGGING_LEVEL_LOGGERS_APP=INFO
# Python
PYTHONPATH=/app
# Mail
DJANGO_EMAIL_HOST=<smtp host>
DJANGO_EMAIL_HOST_USER=<smtp user>
DJANGO_EMAIL_HOST_PASSWORD=<smtp password>
DJANGO_EMAIL_PORT=<smtp port>
DJANGO_EMAIL_FROM=<your email address>
#DJANGO_EMAIL_USE_TLS=true # A flag to enable or disable TLS for email sending.
#DJANGO_EMAIL_USE_SSL=true # A flag to enable or disable SSL for email sending.
DJANGO_EMAIL_BRAND_NAME="La Suite Numérique"
DJANGO_EMAIL_LOGO_IMG="https://${DOCS_HOST}/assets/logo-suite-numerique.png"
# Media
AWS_S3_ENDPOINT_URL=https://${S3_HOST}
AWS_S3_ACCESS_KEY_ID=<s3 access key>
AWS_S3_SECRET_ACCESS_KEY=<s3 secret key>
AWS_STORAGE_BUCKET_NAME=${BUCKET_NAME}
MEDIA_BASE_URL=https://${DOCS_HOST}
# OIDC
OIDC_OP_JWKS_ENDPOINT=https://${KEYCLOAK_HOST}/realms/${REALM_NAME}/protocol/openid-connect/certs
OIDC_OP_AUTHORIZATION_ENDPOINT=https://${KEYCLOAK_HOST}/realms/${REALM_NAME}/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT=https://${KEYCLOAK_HOST}/realms/${REALM_NAME}/protocol/openid-connect/token
OIDC_OP_USER_ENDPOINT=https://${KEYCLOAK_HOST}/realms/${REALM_NAME}/protocol/openid-connect/userinfo
OIDC_OP_LOGOUT_ENDPOINT=https://${KEYCLOAK_HOST}/realms/${REALM_NAME}/protocol/openid-connect/logout
OIDC_RP_CLIENT_ID=<client_id>
OIDC_RP_CLIENT_SECRET=<client secret>
OIDC_RP_SIGN_ALGO=RS256
OIDC_RP_SCOPES="openid email"
#USER_OIDC_FIELD_TO_SHORTNAME
#USER_OIDC_FIELDS_TO_FULLNAME
LOGIN_REDIRECT_URL=https://${DOCS_HOST}
LOGIN_REDIRECT_URL_FAILURE=https://${DOCS_HOST}
LOGOUT_REDIRECT_URL=https://${DOCS_HOST}
OIDC_REDIRECT_ALLOWED_HOSTS=["https://${DOCS_HOST}"]
# AI
#AI_FEATURE_ENABLED=true # is false by default
#AI_BASE_URL=https://openaiendpoint.com
#AI_API_KEY=<API key>
#AI_MODEL=<model used> e.g. llama
# Frontend
#FRONTEND_THEME=mytheme
#FRONTEND_CSS_URL=https://storage.yourdomain.tld/themes/custom.css
#FRONTEND_FOOTER_FEATURE_ENABLED=true
#FRONTEND_URL_JSON_FOOTER=https://docs.domain.tld/contents/footer-demo.json

View File

@@ -1,9 +0,0 @@
DOCS_HOST=docs.domain.tld
KEYCLOAK_HOST=id.domain.tld
S3_HOST=storage.domain.tld
BACKEND_HOST=backend
FRONTEND_HOST=frontend
YPROVIDER_HOST=y-provider
BUCKET_NAME=docs-media-storage
REALM_NAME=docs
#COLLABORATION_WS_URL=wss://${DOCS_HOST}/collaboration/ws/

View File

@@ -1,13 +0,0 @@
# Postgresql db container configuration
POSTGRES_DB=keycloak
POSTGRES_USER=keycloak
POSTGRES_PASSWORD=<generate postgres password>
PGDATA=/var/lib/postgresql/data/pgdata
# Keycloak postgresql configuration
KC_DB=postgres
KC_DB_SCHEMA=public
KC_DB_HOST=postgresql
KC_DB_NAME=${POSTGRES_DB}
KC_DB_USER=${POSTGRES_USER}
KC_DB_PASSWORD=${POSTGRES_PASSWORD}

View File

@@ -1,8 +0,0 @@
# Keycloak admin user
KC_BOOTSTRAP_ADMIN_USERNAME=admin
KC_BOOTSTRAP_ADMIN_PASSWORD=<generate your password>
# Keycloak configuration
KC_HOSTNAME=https://id.yourdomain.tld # Change with your own URL
KC_PROXY_HEADERS=xforwarded # in this example we are running behind an nginx proxy
KC_HTTP_ENABLED=true # in this example we are running behind an nginx proxy

View File

@@ -1,11 +0,0 @@
# App database configuration
DB_HOST=postgresql
DB_NAME=docs
DB_USER=docs
DB_PASSWORD=<generate a secure password>
DB_PORT=5432
# Postgresql db container configuration
POSTGRES_DB=docs
POSTGRES_USER=docs
POSTGRES_PASSWORD=${DB_PASSWORD}

View File

@@ -1,7 +0,0 @@
Y_PROVIDER_API_BASE_URL=http://${YPROVIDER_HOST}:4444/api
Y_PROVIDER_API_KEY=<generate a random key>
COLLABORATION_SERVER_SECRET=<generate a random key>
COLLABORATION_SERVER_ORIGIN=https://${DOCS_HOST}
COLLABORATION_API_URL=https://${DOCS_HOST}/collaboration/api/
COLLABORATION_BACKEND_BASE_URL=https://${DOCS_HOST}
COLLABORATION_LOGGING=true

View File

@@ -24,14 +24,14 @@ class GitmojiTitle(LineRule):
def validate(self, title, _commit):
"""
Download the list possible gitmojis from the project's github repository and check that
Download the list possible gitmojis from the project's GitHub repository and check that
title contains one of them.
"""
gitmojis = requests.get(
"https://raw.githubusercontent.com/carloscuesta/gitmoji/master/packages/gitmojis/src/gitmojis.json"
).json()["gitmojis"]
emojis = [item["emoji"] for item in gitmojis]
pattern = r"^({:s})\(.*\)\s[a-zA-Z].*$".format("|".join(emojis))
pattern = r"^({:s})\(.*\)\s[a-z].*$".format("|".join(emojis))
if not re.search(pattern, title):
violation_msg = 'Title does not match regex "<gitmoji>(<scope>) <subject>"'
return [RuleViolation(self.id, violation_msg, title)]

View File

@@ -1,27 +0,0 @@
publiccodeYmlVersion: "2.4.0"
name: Docs
url: https://github.com/suitenumerique/docs
landingURL: https://github.com/suitenumerique/docs
creationDate: 2023-12-10
logo: https://raw.githubusercontent.com/suitenumerique/docs/main/docs/assets/docs-logo.png
usedBy:
- Direction interministériel du numérique (DINUM)
fundedBy:
- name: Direction interministériel du numérique (DINUM)
url: https://www.numerique.gouv.fr
roadmap: "https://github.com/orgs/suitenumerique/projects/2/views/1"
softwareType: "standalone/other"
description:
en:
shortDescription: "The open source document editor where your notes can become knowledge through live collaboration"
fr:
shortDescription: "L'éditeur de documents open source où vos notes peuvent devenir des connaissances grâce à la collaboration en direct."
legal:
license: MIT
maintenance:
type: internal
contacts:
- name: "Virgile Deville"
email: "virgile.deville@numerique.gouv.fr"
- name: "samuel.paccoud"
email: "samuel.paccoud@numerique.gouv.fr"

View File

@@ -1,7 +1,7 @@
{
"extends": ["github>numerique-gouv/renovate-configuration"],
"dependencyDashboard": true,
"labels": ["dependencies", "noChangeLog", "automated"],
"labels": ["dependencies", "noChangeLog"],
"packageRules": [
{
"enabled": false,
@@ -9,26 +9,11 @@
"matchManagers": ["pep621"],
"matchPackageNames": []
},
{
"groupName": "allowed redis versions",
"matchManagers": ["pep621"],
"matchPackageNames": ["redis"],
"allowedVersions": "<6.0.0"
},
{
"enabled": false,
"groupName": "ignored js dependencies",
"matchManagers": ["npm"],
"matchPackageNames": [
"@hocuspocus/provider",
"@hocuspocus/server",
"docx",
"eslint",
"fetch-mock",
"node",
"node-fetch",
"workbox-webpack-plugin"
]
"matchPackageNames": ["node", "node-fetch", "i18next-parser", "eslint"]
}
]
}

View File

View File

1
secrets Submodule

Submodule secrets added at d7cfe7bcdc

View File

@@ -172,7 +172,7 @@ ignore-on-opaque-inference=yes
# for classes with dynamically set attributes). This supports the use of
# qualified names.
ignored-classes=optparse.Values,thread._local,_thread._local,responses,
Template,Contact
Team,Contact
# List of module names for which member attributes should not be checked
# (useful for modules/projects where namespaces are manipulated during runtime
@@ -447,10 +447,10 @@ max-bool-expr=5
max-branches=12
# Maximum number of locals for function / method body
max-locals=20
max-locals=15
# Maximum number of parents for a class (see R0901).
max-parents=10
max-parents=7
# Maximum number of public methods for a class (see R0904).
max-public-methods=20

View File

@@ -1,3 +1,3 @@
include LICENSE
include README.md
recursive-include src/backend/impress *.html *.png *.gif *.css *.ico *.jpg *.jpeg *.po *.mo *.eot *.svg *.ttf *.woff *.woff2
recursive-include src/backend/people *.html *.png *.gif *.css *.ico *.jpg *.jpeg *.po *.mo *.eot *.svg *.ttf *.woff *.woff2

View File

@@ -1,21 +1,65 @@
"""Admin classes and registrations for core app."""
"""Admin classes and registrations for People's core app."""
from django import forms
from django.contrib import admin
from django.contrib.auth import admin as auth_admin
from django.utils.translation import gettext_lazy as _
from treebeard.admin import TreeAdmin
from treebeard.forms import movenodeform_factory
from . import models
class TemplateAccessInline(admin.TabularInline):
"""Inline admin class for template accesses."""
class IdentityFormSet(forms.BaseInlineFormSet):
"""
Make the "is_main" field readonly when it is True so that declaring another identity
works in the admin.
"""
autocomplete_fields = ["user"]
model = models.TemplateAccess
def add_fields(self, form, index):
"""Disable the "is_main" field when it is set to True"""
super().add_fields(form, index)
is_main_value = form.instance.is_main if form.instance else False
form.fields["is_main"].disabled = is_main_value
class IdentityInline(admin.TabularInline):
"""Inline admin class for user identities."""
fields = (
"sub",
"email",
"is_main",
"created_at",
"updated_at",
)
formset = IdentityFormSet
model = models.Identity
extra = 0
readonly_fields = ("email", "created_at", "sub", "updated_at")
def has_add_permission(self, request, obj):
"""
Identities are automatically created on successful OIDC logins.
Disable creating identities via the admin.
"""
return False
class TeamAccessInline(admin.TabularInline):
"""Inline admin class for team accesses."""
extra = 0
autocomplete_fields = ["user", "team"]
model = models.TeamAccess
readonly_fields = ("created_at", "updated_at")
class TeamWebhookInline(admin.TabularInline):
"""Inline admin class for team webhooks."""
extra = 0
autocomplete_fields = ["team"]
model = models.TeamWebhook
readonly_fields = ("created_at", "updated_at")
@admin.register(models.User)
@@ -28,24 +72,11 @@ class UserAdmin(auth_admin.UserAdmin):
{
"fields": (
"id",
"admin_email",
"password",
)
},
),
(
_("Personal info"),
{
"fields": (
"sub",
"email",
"full_name",
"short_name",
"language",
"timezone",
)
},
),
(_("Personal info"), {"fields": ("admin_email", "language", "timezone")}),
(
_("Permissions"),
{
@@ -66,117 +97,38 @@ class UserAdmin(auth_admin.UserAdmin):
None,
{
"classes": ("wide",),
"fields": ("email", "password1", "password2"),
"fields": ("admin_email", "password1", "password2"),
},
),
)
inlines = (TemplateAccessInline,)
inlines = (IdentityInline, TeamAccessInline)
list_display = (
"id",
"sub",
"full_name",
"admin_email",
"email",
"created_at",
"updated_at",
"is_active",
"is_device",
"is_staff",
"is_superuser",
"is_device",
"created_at",
"updated_at",
)
list_filter = ("is_staff", "is_superuser", "is_device", "is_active")
ordering = (
"is_active",
"-is_superuser",
"-is_staff",
"-is_device",
"-updated_at",
"full_name",
)
readonly_fields = (
"id",
"sub",
"email",
"full_name",
"short_name",
"created_at",
"updated_at",
)
search_fields = ("id", "sub", "admin_email", "email", "full_name")
ordering = ("is_active", "-is_superuser", "-is_staff", "-is_device", "-updated_at")
readonly_fields = ("id", "created_at", "updated_at")
search_fields = ("id", "admin_email", "identities__sub", "identities__email")
@admin.register(models.Template)
class TemplateAdmin(admin.ModelAdmin):
"""Template admin interface declaration."""
@admin.register(models.Team)
class TeamAdmin(admin.ModelAdmin):
"""Team admin interface declaration."""
inlines = (TemplateAccessInline,)
class DocumentAccessInline(admin.TabularInline):
"""Inline admin class for template accesses."""
autocomplete_fields = ["user"]
model = models.DocumentAccess
extra = 0
@admin.register(models.Document)
class DocumentAdmin(TreeAdmin):
"""Document admin interface declaration."""
fieldsets = (
(
None,
{
"fields": (
"id",
"title",
)
},
),
(
_("Permissions"),
{
"fields": (
"creator",
"link_reach",
"link_role",
)
},
),
(
_("Tree structure"),
{
"fields": (
"path",
"depth",
"numchild",
"duplicated_from",
"attachments",
)
},
),
)
form = movenodeform_factory(models.Document)
inlines = (DocumentAccessInline,)
inlines = (TeamAccessInline, TeamWebhookInline)
list_display = (
"id",
"title",
"link_reach",
"link_role",
"name",
"slug",
"created_at",
"updated_at",
)
readonly_fields = (
"attachments",
"creator",
"depth",
"duplicated_from",
"id",
"numchild",
"path",
)
search_fields = ("id", "title")
search_fields = ("name",)
@admin.register(models.Invitation)
@@ -185,7 +137,7 @@ class InvitationAdmin(admin.ModelAdmin):
fields = (
"email",
"document",
"team",
"role",
"created_at",
"issuer",
@@ -197,11 +149,25 @@ class InvitationAdmin(admin.ModelAdmin):
)
list_display = (
"email",
"document",
"team",
"created_at",
"is_expired",
)
def get_readonly_fields(self, request, obj=None):
if obj:
# all fields read only = disable update
return self.fields
return self.readonly_fields
def change_view(self, request, object_id, form_url="", extra_context=None):
"""Custom edit form. Remove 'save' buttons."""
extra_context = extra_context or {}
extra_context["show_save_and_continue"] = False
extra_context["show_save"] = False
extra_context["show_save_and_add_another"] = False
return super().change_view(request, object_id, extra_context=extra_context)
def save_model(self, request, obj, form, change):
obj.issuer = request.user
obj.save()

View File

@@ -1,4 +1,4 @@
"""Impress core API endpoints"""
"""People core API endpoints"""
from django.conf import settings
from django.core.exceptions import ValidationError
@@ -17,13 +17,14 @@ def exception_handler(exc, context):
https://gist.github.com/twidi/9d55486c36b6a51bdcb05ce3a763e79f
"""
if isinstance(exc, ValidationError):
detail = None
if hasattr(exc, "message_dict"):
detail = exc.message_dict
elif hasattr(exc, "message"):
detail = exc.message
elif hasattr(exc, "messages"):
detail = exc.messages
else:
detail = ""
exc = drf_exceptions.ValidationError(detail=detail)

View File

@@ -1,25 +0,0 @@
"""A JSONField for DRF to handle serialization/deserialization."""
import json
from rest_framework import serializers
class JSONField(serializers.Field):
"""
A custom field for handling JSON data.
"""
def to_representation(self, value):
"""
Convert the JSON string to a Python dictionary for serialization.
"""
return value
def to_internal_value(self, data):
"""
Convert the Python dictionary to a JSON string for deserialization.
"""
if data is None:
return None
return json.dumps(data)

View File

@@ -1,108 +0,0 @@
"""API filters for Impress' core application."""
import unicodedata
from django.utils.translation import gettext_lazy as _
import django_filters
from core import models
def remove_accents(value):
"""Remove accents from a string (vélo -> velo)."""
return "".join(
c
for c in unicodedata.normalize("NFD", value)
if unicodedata.category(c) != "Mn"
)
class AccentInsensitiveCharFilter(django_filters.CharFilter):
"""
A custom CharFilter that filters on the accent-insensitive value searched.
"""
def filter(self, qs, value):
"""
Apply the filter to the queryset using the unaccented version of the field.
Args:
qs: The queryset to filter.
value: The value to search for in the unaccented field.
Returns:
A filtered queryset.
"""
if value:
value = remove_accents(value)
return super().filter(qs, value)
class DocumentFilter(django_filters.FilterSet):
"""
Custom filter for filtering documents on title (accent and case insensitive).
"""
title = AccentInsensitiveCharFilter(
field_name="title", lookup_expr="unaccent__icontains", label=_("Title")
)
class Meta:
model = models.Document
fields = ["title"]
class ListDocumentFilter(DocumentFilter):
"""
Custom filter for filtering documents.
"""
is_creator_me = django_filters.BooleanFilter(
method="filter_is_creator_me", label=_("Creator is me")
)
is_favorite = django_filters.BooleanFilter(
method="filter_is_favorite", label=_("Favorite")
)
class Meta:
model = models.Document
fields = ["is_creator_me", "is_favorite", "title"]
# pylint: disable=unused-argument
def filter_is_creator_me(self, queryset, name, value):
"""
Filter documents based on the `creator` being the current user.
Example:
- /api/v1.0/documents/?is_creator_me=true
→ Filters documents created by the logged-in user
- /api/v1.0/documents/?is_creator_me=false
→ Filters documents created by other users
"""
user = self.request.user
if not user.is_authenticated:
return queryset
if value:
return queryset.filter(creator=user)
return queryset.exclude(creator=user)
# pylint: disable=unused-argument
def filter_is_favorite(self, queryset, name, value):
"""
Filter documents based on whether they are marked as favorite by the current user.
Example:
- /api/v1.0/documents/?is_favorite=true
→ Filters documents marked as favorite by the logged-in user
- /api/v1.0/documents/?is_favorite=false
→ Filters documents not marked as favorite by the logged-in user
"""
user = self.request.user
if not user.is_authenticated:
return queryset
return queryset.filter(is_favorite=bool(value))

View File

@@ -1,19 +1,9 @@
"""Permission handlers for the impress core app."""
"""Permission handlers for the People core app."""
from django.core import exceptions
from django.db.models import Q
from django.http import Http404
from rest_framework import permissions
from core import choices
from core.models import DocumentAccess, RoleChoices, get_trashbin_cutoff
ACTION_FOR_METHOD_TO_PERMISSION = {
"versions_detail": {"DELETE": "versions_destroy", "GET": "versions_retrieve"},
"children": {"GET": "children_list", "POST": "children_create"},
}
class IsAuthenticated(permissions.BasePermission):
"""
@@ -22,16 +12,7 @@ class IsAuthenticated(permissions.BasePermission):
"""
def has_permission(self, request, view):
return bool(request.auth) or request.user.is_authenticated
class IsAuthenticatedOrSafe(IsAuthenticated):
"""Allows access to authenticated users (or anonymous users but only on safe methods)."""
def has_permission(self, request, view):
if request.method in permissions.SAFE_METHODS:
return True
return super().has_permission(request, view)
return bool(request.auth) if request.auth else request.user.is_authenticated
class IsSelf(IsAuthenticated):
@@ -65,109 +46,10 @@ class IsOwnedOrPublic(IsAuthenticated):
return False
class CanCreateInvitationPermission(permissions.BasePermission):
"""
Custom permission class to handle permission checks for managing invitations.
"""
def has_permission(self, request, view):
user = request.user
# Ensure the user is authenticated
if not (bool(request.auth) or request.user.is_authenticated):
return False
# Apply permission checks only for creation (POST requests)
if view.action != "create":
return True
# Check if resource_id is passed in the context
try:
document_id = view.kwargs["resource_id"]
except KeyError as exc:
raise exceptions.ValidationError(
"You must set a document ID in kwargs to manage document invitations."
) from exc
# Check if the user has access to manage invitations (Owner/Admin roles)
return DocumentAccess.objects.filter(
Q(user=user) | Q(team__in=user.teams),
document=document_id,
role__in=[RoleChoices.OWNER, RoleChoices.ADMIN],
).exists()
class ResourceWithAccessPermission(permissions.BasePermission):
"""A permission class for templates and invitations."""
def has_permission(self, request, view):
"""check create permission for templates."""
return request.user.is_authenticated or view.action != "create"
class AccessPermission(IsAuthenticated):
"""Permission class for access objects."""
def has_object_permission(self, request, view, obj):
"""Check permission for a given object."""
abilities = obj.get_abilities(request.user)
action = view.action
return abilities.get(action, False)
class DocumentPermission(permissions.BasePermission):
"""Subclass to handle soft deletion specificities."""
def has_permission(self, request, view):
"""check create permission for documents."""
return request.user.is_authenticated or view.action != "create"
def has_object_permission(self, request, view, obj):
"""
Return a 404 on deleted documents
- for which the trashbin cutoff is past
- for which the current user is not owner of the document or one of its ancestors
"""
if (
deleted_at := obj.ancestors_deleted_at
) and deleted_at < get_trashbin_cutoff():
raise Http404
abilities = obj.get_abilities(request.user)
action = view.action
try:
action = ACTION_FOR_METHOD_TO_PERMISSION[view.action][request.method]
except KeyError:
pass
has_permission = abilities.get(action, False)
if obj.ancestors_deleted_at and not RoleChoices.OWNER in obj.user_roles:
raise Http404
return has_permission
class ResourceAccessPermission(IsAuthenticated):
"""Permission class for document access objects."""
def has_permission(self, request, view):
"""check create permission for accesses in documents tree."""
if super().has_permission(request, view) is False:
return False
if view.action == "create":
role = getattr(view, view.resource_field_name).get_role(request.user)
if role not in choices.PRIVILEGED_ROLES:
raise exceptions.PermissionDenied(
"You are not allowed to manage accesses for this resource."
)
return True
def has_object_permission(self, request, view, obj):
"""Check permission for a given object."""
abilities = obj.get_abilities(request.user)
requested_role = request.data.get("role")
if requested_role and requested_role not in abilities.get("set_role_to", []):
return False
action = view.action
return abilities.get(action, False)
return abilities.get(request.method.lower(), False)

View File

@@ -1,803 +1,239 @@
"""Client serializers for the impress core app."""
"""Client serializers for the People core app."""
import binascii
import mimetypes
from base64 import b64decode
from rest_framework import exceptions, serializers
from timezone_field.rest_framework import TimeZoneSerializerField
from django.conf import settings
from django.db.models import Q
from django.utils.functional import lazy
from django.utils.translation import gettext_lazy as _
import magic
from rest_framework import serializers
from core import choices, enums, models, utils
from core.services.ai_services import AI_ACTIONS
from core.services.converter_services import (
ConversionError,
YdocConverter,
)
from core import models
class UserSerializer(serializers.ModelSerializer):
class ContactSerializer(serializers.ModelSerializer):
"""Serialize contacts."""
class Meta:
model = models.Contact
fields = [
"id",
"base",
"data",
"full_name",
"owner",
"short_name",
]
read_only_fields = ["id", "owner"]
def update(self, instance, validated_data):
"""Make "base" field readonly but only for update/patch."""
validated_data.pop("base", None)
return super().update(instance, validated_data)
class DynamicFieldsModelSerializer(serializers.ModelSerializer):
"""
A ModelSerializer that takes an additional `fields` argument that
controls which fields should be displayed.
"""
def __init__(self, *args, **kwargs):
# Don't pass the 'fields' arg up to the superclass
fields = kwargs.pop("fields", None)
# Instantiate the superclass normally
super().__init__(*args, **kwargs)
if fields is not None:
# Drop any fields that are not specified in the `fields` argument.
allowed = set(fields)
existing = set(self.fields)
for field_name in existing - allowed:
self.fields.pop(field_name)
class UserSerializer(DynamicFieldsModelSerializer):
"""Serialize users."""
class Meta:
model = models.User
fields = ["id", "email", "full_name", "short_name", "language"]
read_only_fields = ["id", "email", "full_name", "short_name"]
class UserLightSerializer(UserSerializer):
"""Serialize users with limited fields."""
timezone = TimeZoneSerializerField(use_pytz=False, required=True)
email = serializers.ReadOnlyField()
name = serializers.ReadOnlyField()
class Meta:
model = models.User
fields = ["full_name", "short_name"]
read_only_fields = ["full_name", "short_name"]
fields = [
"id",
"email",
"language",
"name",
"timezone",
"is_device",
"is_staff",
]
read_only_fields = ["id", "name", "email", "is_device", "is_staff"]
class TemplateAccessSerializer(serializers.ModelSerializer):
"""Serialize template accesses."""
class TeamAccessSerializer(serializers.ModelSerializer):
"""Serialize team accesses."""
abilities = serializers.SerializerMethodField(read_only=True)
class Meta:
model = models.TemplateAccess
resource_field_name = "template"
fields = ["id", "user", "team", "role", "abilities"]
model = models.TeamAccess
fields = ["id", "user", "role", "abilities"]
read_only_fields = ["id", "abilities"]
def get_abilities(self, instance) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return instance.get_abilities(request.user)
return {}
def update(self, instance, validated_data):
"""Make "user" field is readonly but only on update."""
validated_data.pop("user", None)
return super().update(instance, validated_data)
class ListDocumentSerializer(serializers.ModelSerializer):
"""Serialize documents with limited fields for display in lists."""
is_favorite = serializers.BooleanField(read_only=True)
nb_accesses_ancestors = serializers.IntegerField(read_only=True)
nb_accesses_direct = serializers.IntegerField(read_only=True)
user_role = serializers.SerializerMethodField(read_only=True)
abilities = serializers.SerializerMethodField(read_only=True)
class Meta:
model = models.Document
fields = [
"id",
"abilities",
"ancestors_link_reach",
"ancestors_link_role",
"computed_link_reach",
"computed_link_role",
"created_at",
"creator",
"depth",
"excerpt",
"is_favorite",
"link_role",
"link_reach",
"nb_accesses_ancestors",
"nb_accesses_direct",
"numchild",
"path",
"title",
"updated_at",
"user_role",
]
read_only_fields = [
"id",
"abilities",
"ancestors_link_reach",
"ancestors_link_role",
"computed_link_reach",
"computed_link_role",
"created_at",
"creator",
"depth",
"excerpt",
"is_favorite",
"link_role",
"link_reach",
"nb_accesses_ancestors",
"nb_accesses_direct",
"numchild",
"path",
"updated_at",
"user_role",
]
def to_representation(self, instance):
"""Precompute once per instance"""
paths_links_mapping = self.context.get("paths_links_mapping")
if paths_links_mapping is not None:
links = paths_links_mapping.get(instance.path[: -instance.steplen], [])
instance.ancestors_link_definition = choices.get_equivalent_link_definition(
links
)
return super().to_representation(instance)
def get_abilities(self, instance) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if not request:
return {}
return instance.get_abilities(request.user)
def get_user_role(self, instance):
"""
Return roles of the logged-in user for the current document,
taking into account ancestors.
"""
request = self.context.get("request")
return instance.get_role(request.user) if request else None
class DocumentLightSerializer(serializers.ModelSerializer):
"""Minial document serializer for nesting in document accesses."""
class Meta:
model = models.Document
fields = ["id", "path", "depth"]
read_only_fields = ["id", "path", "depth"]
class DocumentSerializer(ListDocumentSerializer):
"""Serialize documents with all fields for display in detail views."""
content = serializers.CharField(required=False)
class Meta:
model = models.Document
fields = [
"id",
"abilities",
"ancestors_link_reach",
"ancestors_link_role",
"computed_link_reach",
"computed_link_role",
"content",
"created_at",
"creator",
"depth",
"excerpt",
"is_favorite",
"link_role",
"link_reach",
"nb_accesses_ancestors",
"nb_accesses_direct",
"numchild",
"path",
"title",
"updated_at",
"user_role",
]
read_only_fields = [
"id",
"abilities",
"ancestors_link_reach",
"ancestors_link_role",
"computed_link_reach",
"computed_link_role",
"created_at",
"creator",
"depth",
"is_favorite",
"link_role",
"link_reach",
"nb_accesses_ancestors",
"nb_accesses_direct",
"numchild",
"path",
"updated_at",
"user_role",
]
def get_fields(self):
"""Dynamically make `id` read-only on PUT requests but writable on POST requests."""
fields = super().get_fields()
request = self.context.get("request")
if request and request.method == "POST":
fields["id"].read_only = False
return fields
def validate_id(self, value):
"""Ensure the provided ID does not already exist when creating a new document."""
request = self.context.get("request")
# Only check this on POST (creation)
if request and request.method == "POST":
if models.Document.objects.filter(id=value).exists():
raise serializers.ValidationError(
"A document with this ID already exists. You cannot override it."
)
return value
def validate_content(self, value):
"""Validate the content field."""
if not value:
return None
try:
b64decode(value, validate=True)
except binascii.Error as err:
raise serializers.ValidationError("Invalid base64 content.") from err
return value
def save(self, **kwargs):
"""
Process the content field to extract attachment keys and update the document's
"attachments" field for access control.
"""
content = self.validated_data.get("content", "")
extracted_attachments = set(utils.extract_attachments(content))
existing_attachments = (
set(self.instance.attachments or []) if self.instance else set()
)
new_attachments = extracted_attachments - existing_attachments
if new_attachments:
attachments_documents = (
models.Document.objects.filter(
attachments__overlap=list(new_attachments)
)
.only("path", "attachments")
.order_by("path")
)
user = self.context["request"].user
readable_per_se_paths = (
models.Document.objects.readable_per_se(user)
.order_by("path")
.values_list("path", flat=True)
)
readable_attachments_paths = utils.filter_descendants(
[doc.path for doc in attachments_documents],
readable_per_se_paths,
skip_sorting=True,
)
readable_attachments = set()
for document in attachments_documents:
if document.path not in readable_attachments_paths:
continue
readable_attachments.update(set(document.attachments) & new_attachments)
# Update attachments with readable keys
self.validated_data["attachments"] = list(
existing_attachments | readable_attachments
)
return super().save(**kwargs)
class DocumentAccessSerializer(serializers.ModelSerializer):
"""Serialize document accesses."""
document = DocumentLightSerializer(read_only=True)
user_id = serializers.PrimaryKeyRelatedField(
queryset=models.User.objects.all(),
write_only=True,
source="user",
required=False,
allow_null=True,
)
user = UserSerializer(read_only=True)
team = serializers.CharField(required=False, allow_blank=True)
abilities = serializers.SerializerMethodField(read_only=True)
max_ancestors_role = serializers.SerializerMethodField(read_only=True)
max_role = serializers.SerializerMethodField(read_only=True)
class Meta:
model = models.DocumentAccess
resource_field_name = "document"
fields = [
"id",
"document",
"user",
"user_id",
"team",
"role",
"abilities",
"max_ancestors_role",
"max_role",
]
read_only_fields = [
"id",
"document",
"abilities",
"max_ancestors_role",
"max_role",
]
def get_abilities(self, instance) -> dict:
def get_abilities(self, access) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return instance.get_abilities(request.user)
return access.get_abilities(request.user)
return {}
def get_max_ancestors_role(self, instance):
"""Return max_ancestors_role if annotated; else None."""
return getattr(instance, "max_ancestors_role", None)
def get_max_role(self, instance):
"""Return max_ancestors_role if annotated; else None."""
return choices.RoleChoices.max(
getattr(instance, "max_ancestors_role", None),
instance.role,
)
def update(self, instance, validated_data):
"""Make "user" field readonly but only on update."""
validated_data.pop("team", None)
validated_data.pop("user", None)
return super().update(instance, validated_data)
class DocumentAccessLightSerializer(DocumentAccessSerializer):
"""Serialize document accesses with limited fields."""
user = UserLightSerializer(read_only=True)
class Meta:
model = models.DocumentAccess
resource_field_name = "document"
fields = [
"id",
"document",
"user",
"team",
"role",
"abilities",
"max_ancestors_role",
"max_role",
]
read_only_fields = [
"id",
"document",
"team",
"role",
"abilities",
"max_ancestors_role",
"max_role",
]
class ServerCreateDocumentSerializer(serializers.Serializer):
"""
Serializer for creating a document from a server-to-server request.
Expects 'content' as a markdown string, which is converted to our internal format
via a Node.js microservice. The conversion is handled automatically, so third parties
only need to provide markdown.
Both "sub" and "email" are required because the external app calling doesn't know
if the user will pre-exist in Docs database. If the user pre-exist, we will ignore the
submitted "email" field and use the email address set on the user account in our database
"""
# Document
title = serializers.CharField(required=True)
content = serializers.CharField(required=True)
# User
sub = serializers.CharField(
required=True, validators=[models.User.sub_validator], max_length=255
)
email = serializers.EmailField(required=True)
language = serializers.ChoiceField(
required=False, choices=lazy(lambda: settings.LANGUAGES, tuple)()
)
# Invitation
message = serializers.CharField(required=False)
subject = serializers.CharField(required=False)
def create(self, validated_data):
"""Create the document and associate it with the user or send an invitation."""
language = validated_data.get("language", settings.LANGUAGE_CODE)
# Get the user on its sub (unique identifier). Default on email if allowed in settings
email = validated_data["email"]
try:
user = models.User.objects.get_user_by_sub_or_email(
validated_data["sub"], email
)
except models.DuplicateEmailError as err:
raise serializers.ValidationError({"email": [err.message]}) from err
if user:
email = user.email
language = user.language or language
try:
document_content = YdocConverter().convert_markdown(
validated_data["content"]
)
except ConversionError as err:
raise serializers.ValidationError(
{"content": ["Could not convert content"]}
) from err
document = models.Document.add_root(
title=validated_data["title"],
content=document_content,
creator=user,
)
if user:
# Associate the document with the pre-existing user
models.DocumentAccess.objects.create(
document=document,
role=models.RoleChoices.OWNER,
user=user,
)
else:
# The user doesn't exist in our database: we need to invite him/her
models.Invitation.objects.create(
document=document,
email=email,
role=models.RoleChoices.OWNER,
)
self._send_email_notification(document, validated_data, email, language)
return document
def _send_email_notification(self, document, validated_data, email, language):
"""Notify the user about the newly created document."""
subject = validated_data.get("subject") or _(
"A new document was created on your behalf!"
)
context = {
"message": validated_data.get("message")
or _("You have been granted ownership of a new document:"),
"title": subject,
}
document.send_email(subject, [email], context, language)
def update(self, instance, validated_data):
"""
This serializer does not support updates.
"""
raise NotImplementedError("Update is not supported for this serializer.")
class LinkDocumentSerializer(serializers.ModelSerializer):
"""
Serialize link configuration for documents.
We expose it separately from document in order to simplify and secure access control.
"""
class Meta:
model = models.Document
fields = [
"link_role",
"link_reach",
]
class DocumentDuplicationSerializer(serializers.Serializer):
"""
Serializer for duplicating a document.
Allows specifying whether to keep access permissions.
"""
with_accesses = serializers.BooleanField(default=False)
def create(self, validated_data):
"""
This serializer is not intended to create objects.
"""
raise NotImplementedError("This serializer does not support creation.")
def update(self, instance, validated_data):
"""
This serializer is not intended to update objects.
"""
raise NotImplementedError("This serializer does not support updating.")
# Suppress the warning about not implementing `create` and `update` methods
# since we don't use a model and only rely on the serializer for validation
# pylint: disable=abstract-method
class FileUploadSerializer(serializers.Serializer):
"""Receive file upload requests."""
file = serializers.FileField()
def validate_file(self, file):
"""Add file size and type constraints as defined in settings."""
# Validate file size
if file.size > settings.DOCUMENT_IMAGE_MAX_SIZE:
max_size = settings.DOCUMENT_IMAGE_MAX_SIZE // (1024 * 1024)
raise serializers.ValidationError(
f"File size exceeds the maximum limit of {max_size:d} MB."
)
extension = file.name.rpartition(".")[-1] if "." in file.name else None
# Read the first few bytes to determine the MIME type accurately
mime = magic.Magic(mime=True)
magic_mime_type = mime.from_buffer(file.read(1024))
file.seek(0) # Reset file pointer to the beginning after reading
self.context["is_unsafe"] = False
if settings.DOCUMENT_ATTACHMENT_CHECK_UNSAFE_MIME_TYPES_ENABLED:
self.context["is_unsafe"] = (
magic_mime_type in settings.DOCUMENT_UNSAFE_MIME_TYPES
)
extension_mime_type, _ = mimetypes.guess_type(file.name)
# Try guessing a coherent extension from the mimetype
if extension_mime_type != magic_mime_type:
self.context["is_unsafe"] = True
guessed_ext = mimetypes.guess_extension(magic_mime_type)
# Missing extensions or extensions longer than 5 characters (it's as long as an extension
# can be) are replaced by the extension we eventually guessed from mimetype.
if (extension is None or len(extension) > 5) and guessed_ext:
extension = guessed_ext[1:]
if extension is None:
raise serializers.ValidationError("Could not determine file extension.")
self.context["expected_extension"] = extension
self.context["content_type"] = magic_mime_type
self.context["file_name"] = file.name
return file
def validate(self, attrs):
"""Override validate to add the computed extension to validated_data."""
attrs["expected_extension"] = self.context["expected_extension"]
attrs["is_unsafe"] = self.context["is_unsafe"]
attrs["content_type"] = self.context["content_type"]
attrs["file_name"] = self.context["file_name"]
"""
Check access rights specific to writing (create/update)
"""
request = self.context.get("request")
user = getattr(request, "user", None)
role = attrs.get("role")
# Update
if self.instance:
can_set_role_to = self.instance.get_abilities(user)["set_role_to"]
if role and role not in can_set_role_to:
message = (
f"You are only allowed to set role to {', '.join(can_set_role_to)}"
if can_set_role_to
else "You are not allowed to set this role for this team."
)
raise exceptions.PermissionDenied(message)
# Create
else:
try:
team_id = self.context["team_id"]
except KeyError as exc:
raise exceptions.ValidationError(
"You must set a team ID in kwargs to create a new team access."
) from exc
if not models.TeamAccess.objects.filter(
team=team_id,
user=user,
role__in=[models.RoleChoices.OWNER, models.RoleChoices.ADMIN],
).exists():
raise exceptions.PermissionDenied(
"You are not allowed to manage accesses for this team."
)
if (
role == models.RoleChoices.OWNER
and not models.TeamAccess.objects.filter(
team=team_id,
user=user,
role=models.RoleChoices.OWNER,
).exists()
):
raise exceptions.PermissionDenied(
"Only owners of a team can assign other users as owners."
)
attrs["team_id"] = self.context["team_id"]
return attrs
class TemplateSerializer(serializers.ModelSerializer):
"""Serialize templates."""
class TeamAccessReadOnlySerializer(TeamAccessSerializer):
"""Serialize team accesses for list and retrieve actions."""
abilities = serializers.SerializerMethodField(read_only=True)
accesses = TemplateAccessSerializer(many=True, read_only=True)
user = UserSerializer(read_only=True, fields=["id", "name", "email"])
class Meta:
model = models.Template
model = models.TeamAccess
fields = [
"id",
"title",
"user",
"role",
"abilities",
]
read_only_fields = [
"id",
"user",
"role",
"abilities",
]
class TeamSerializer(serializers.ModelSerializer):
"""Serialize teams."""
abilities = serializers.SerializerMethodField(read_only=True)
slug = serializers.SerializerMethodField()
class Meta:
model = models.Team
fields = [
"id",
"name",
"accesses",
"abilities",
"css",
"code",
"is_public",
"slug",
"created_at",
"updated_at",
]
read_only_fields = [
"id",
"accesses",
"abilities",
"slug",
"created_at",
"updated_at",
]
read_only_fields = ["id", "accesses", "abilities"]
def get_abilities(self, document) -> dict:
def get_abilities(self, team) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return document.get_abilities(request.user)
return team.get_abilities(request.user)
return {}
# pylint: disable=abstract-method
class DocumentGenerationSerializer(serializers.Serializer):
"""Serializer to receive a request to generate a document on a template."""
body = serializers.CharField(label=_("Body"))
body_type = serializers.ChoiceField(
choices=["html", "markdown"],
label=_("Body type"),
required=False,
default="html",
)
format = serializers.ChoiceField(
choices=["pdf", "docx"],
label=_("Format"),
required=False,
default="pdf",
)
def get_slug(self, instance):
"""Return slug from the team's name."""
return instance.get_slug()
class InvitationSerializer(serializers.ModelSerializer):
"""Serialize invitations."""
abilities = serializers.SerializerMethodField(read_only=True)
class Meta:
model = models.Invitation
fields = [
"id",
"abilities",
"created_at",
"email",
"document",
"role",
"issuer",
"is_expired",
]
read_only_fields = [
"id",
"abilities",
"created_at",
"document",
"issuer",
"is_expired",
]
def get_abilities(self, invitation) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return invitation.get_abilities(request.user)
return {}
fields = ["id", "created_at", "email", "team", "role", "issuer", "is_expired"]
read_only_fields = ["id", "created_at", "team", "issuer", "is_expired"]
def validate(self, attrs):
"""Validate invitation data."""
"""Validate and restrict invitation to new user based on email."""
request = self.context.get("request")
user = getattr(request, "user", None)
attrs["document_id"] = self.context["resource_id"]
try:
team_id = self.context["team_id"]
except KeyError as exc:
raise exceptions.ValidationError(
"You must set a team ID in kwargs to create a new team invitation."
) from exc
# Only set the issuer if the instance is being created
if self.instance is None:
attrs["issuer"] = user
if not models.TeamAccess.objects.filter(
team=team_id,
user=user,
role__in=[models.RoleChoices.OWNER, models.RoleChoices.ADMIN],
).exists():
raise exceptions.PermissionDenied(
"You are not allowed to manage invitation for this team."
)
attrs["team_id"] = team_id
attrs["issuer"] = user
return attrs
def validate_role(self, role):
"""Custom validation for the role field."""
request = self.context.get("request")
user = getattr(request, "user", None)
document_id = self.context["resource_id"]
# If the role is OWNER, check if the user has OWNER access
if role == models.RoleChoices.OWNER:
if not models.DocumentAccess.objects.filter(
Q(user=user) | Q(team__in=user.teams),
document=document_id,
role=models.RoleChoices.OWNER,
).exists():
raise serializers.ValidationError(
"Only owners of a document can invite other users as owners."
)
return role
class RoleSerializer(serializers.Serializer):
"""Serializer validating role choices."""
role = serializers.ChoiceField(
choices=models.RoleChoices.choices, required=False, allow_null=True
)
class DocumentAskForAccessCreateSerializer(serializers.Serializer):
"""Serializer for creating a document ask for access."""
role = serializers.ChoiceField(
choices=models.RoleChoices.choices,
required=False,
default=models.RoleChoices.READER,
)
class DocumentAskForAccessSerializer(serializers.ModelSerializer):
"""Serializer for document ask for access model"""
abilities = serializers.SerializerMethodField(read_only=True)
user = UserSerializer(read_only=True)
class Meta:
model = models.DocumentAskForAccess
fields = [
"id",
"document",
"user",
"role",
"created_at",
"abilities",
]
read_only_fields = ["id", "document", "user", "role", "created_at", "abilities"]
def get_abilities(self, invitation) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return invitation.get_abilities(request.user)
return {}
class VersionFilterSerializer(serializers.Serializer):
"""Validate version filters applied to the list endpoint."""
version_id = serializers.CharField(required=False, allow_blank=True)
page_size = serializers.IntegerField(
required=False, min_value=1, max_value=50, default=20
)
class AITransformSerializer(serializers.Serializer):
"""Serializer for AI transform requests."""
action = serializers.ChoiceField(choices=AI_ACTIONS, required=True)
text = serializers.CharField(required=True)
def validate_text(self, value):
"""Ensure the text field is not empty."""
if len(value.strip()) == 0:
raise serializers.ValidationError("Text field cannot be empty.")
return value
class AITranslateSerializer(serializers.Serializer):
"""Serializer for AI translate requests."""
language = serializers.ChoiceField(
choices=tuple(enums.ALL_LANGUAGES.items()), required=True
)
text = serializers.CharField(required=True)
def validate_text(self, value):
"""Ensure the text field is not empty."""
if len(value.strip()) == 0:
raise serializers.ValidationError("Text field cannot be empty.")
return value
class MoveDocumentSerializer(serializers.Serializer):
"""
Serializer for validating input data to move a document within the tree structure.
Fields:
- target_document_id (UUIDField): The ID of the target parent document where the
document should be moved. This field is required and must be a valid UUID.
- position (ChoiceField): Specifies the position of the document in relation to
the target parent's children.
Choices:
- "first-child": Place the document as the first child of the target parent.
- "last-child": Place the document as the last child of the target parent (default).
- "left": Place the document as the left sibling of the target parent.
- "right": Place the document as the right sibling of the target parent.
Example:
Input payload for moving a document:
{
"target_document_id": "123e4567-e89b-12d3-a456-426614174000",
"position": "first-child"
}
Notes:
- The `target_document_id` is mandatory.
- The `position` defaults to "last-child" if not provided.
"""
target_document_id = serializers.UUIDField(required=True)
position = serializers.ChoiceField(
choices=enums.MoveNodePositionChoices.choices,
default=enums.MoveNodePositionChoices.LAST_CHILD,
)

View File

@@ -1,181 +0,0 @@
"""Util to generate S3 authorization headers for object storage access control"""
import time
from abc import ABC, abstractmethod
from django.conf import settings
from django.core.cache import cache
from django.core.files.storage import default_storage
import botocore
from rest_framework.throttling import BaseThrottle
def nest_tree(flat_list, steplen):
"""
Convert a flat list of serialized documents into a nested tree making advantage
of the`path` field and its step length.
"""
node_dict = {}
roots = []
# Sort the flat list by path to ensure parent nodes are processed first
flat_list.sort(key=lambda x: x["path"])
for node in flat_list:
node["children"] = [] # Initialize children list
node_dict[node["path"]] = node
# Determine parent path
parent_path = node["path"][:-steplen]
if parent_path in node_dict:
node_dict[parent_path]["children"].append(node)
else:
roots.append(node) # Collect root nodes
if len(roots) > 1:
raise ValueError("More than one root element detected.")
return roots[0] if roots else None
def filter_root_paths(paths, skip_sorting=False):
"""
Filters root paths from a list of paths representing a tree structure.
A root path is defined as a path that is not a prefix of any other path.
Args:
paths (list of str): The list of paths.
Returns:
list of str: The filtered list of root paths.
"""
if not skip_sorting:
paths.sort()
root_paths = []
for path in paths:
# If the current path is not a prefix of the last added root path, add it
if not root_paths or not path.startswith(root_paths[-1]):
root_paths.append(path)
return root_paths
def generate_s3_authorization_headers(key):
"""
Generate authorization headers for an s3 object.
These headers can be used as an alternative to signed urls with many benefits:
- the urls of our files never expire and can be stored in our documents' content
- we don't leak authorized urls that could be shared (file access can only be done
with cookies)
- access control is truly realtime
- the object storage service does not need to be exposed on internet
"""
url = default_storage.unsigned_connection.meta.client.generate_presigned_url(
"get_object",
ExpiresIn=0,
Params={"Bucket": default_storage.bucket_name, "Key": key},
)
request = botocore.awsrequest.AWSRequest(method="get", url=url)
s3_client = default_storage.connection.meta.client
# pylint: disable=protected-access
credentials = s3_client._request_signer._credentials # noqa: SLF001
frozen_credentials = credentials.get_frozen_credentials()
region = s3_client.meta.region_name
auth = botocore.auth.S3SigV4Auth(frozen_credentials, "s3", region)
auth.add_auth(request)
return request
class AIBaseRateThrottle(BaseThrottle, ABC):
"""Base throttle class for AI-related rate limiting with backoff."""
def __init__(self, rates):
"""Initialize instance attributes with configurable rates."""
super().__init__()
self.rates = rates
self.cache_key = None
self.recent_requests_minute = 0
self.recent_requests_hour = 0
self.recent_requests_day = 0
@abstractmethod
def get_cache_key(self, request, view):
"""Abstract method to generate cache key for throttling."""
def allow_request(self, request, view):
"""Check if the request is allowed based on rate limits."""
self.cache_key = self.get_cache_key(request, view)
if not self.cache_key:
return True # Allow if no cache key is generated
now = time.time()
history = cache.get(self.cache_key, [])
# Keep requests within the last 24 hours
history = [req for req in history if req > now - 86400]
# Calculate recent requests
self.recent_requests_minute = len([req for req in history if req > now - 60])
self.recent_requests_hour = len([req for req in history if req > now - 3600])
self.recent_requests_day = len(history)
# Check rate limits
if self.recent_requests_minute >= self.rates["minute"]:
return False
if self.recent_requests_hour >= self.rates["hour"]:
return False
if self.recent_requests_day >= self.rates["day"]:
return False
# Log the request
history.append(now)
cache.set(self.cache_key, history, timeout=86400)
return True
def wait(self):
"""Implement a backoff strategy by increasing wait time based on limits hit."""
if self.recent_requests_day >= self.rates["day"]:
return 86400
if self.recent_requests_hour >= self.rates["hour"]:
return 3600
if self.recent_requests_minute >= self.rates["minute"]:
return 60
return None
class AIDocumentRateThrottle(AIBaseRateThrottle):
"""Throttle for limiting AI requests per document with backoff."""
def __init__(self, *args, **kwargs):
super().__init__(settings.AI_DOCUMENT_RATE_THROTTLE_RATES)
def get_cache_key(self, request, view):
"""Include document ID in the cache key."""
document_id = view.kwargs["pk"]
return f"document_{document_id}_throttle_ai"
class AIUserRateThrottle(AIBaseRateThrottle):
"""Throttle that limits requests per user or IP with backoff and rate limits."""
def __init__(self, *args, **kwargs):
super().__init__(settings.AI_USER_RATE_THROTTLE_RATES)
def get_cache_key(self, request, view=None):
"""Generate a cache key based on the user ID or IP for anonymous users."""
if request.user.is_authenticated:
return f"user_{request.user.id!s}_throttle_ai"
return f"anonymous_{self.get_ident(request)}_throttle_ai"
def get_ident(self, request):
"""Return the request IP address."""
x_forwarded_for = request.META.get("HTTP_X_FORWARDED_FOR")
return (
x_forwarded_for.split(",")[0]
if x_forwarded_for
else request.META.get("REMOTE_ADDR")
)

File diff suppressed because it is too large Load Diff

View File

@@ -1,11 +1,11 @@
"""Impress Core application"""
"""People Core application"""
# from django.apps import AppConfig
# from django.utils.translation import gettext_lazy as _
# class CoreConfig(AppConfig):
# """Configuration class for the impress core app."""
# """Configuration class for the People core app."""
# name = "core"
# app_label = "core"
# verbose_name = _("impress core application")
# verbose_name = _("People core application")

View File

@@ -1,52 +0,0 @@
"""Custom authentication classes for the Impress core app"""
from django.conf import settings
from rest_framework.authentication import BaseAuthentication
from rest_framework.exceptions import AuthenticationFailed
class ServerToServerAuthentication(BaseAuthentication):
"""
Custom authentication class for server-to-server requests.
Validates the presence and correctness of the Authorization header.
"""
AUTH_HEADER = "Authorization"
TOKEN_TYPE = "Bearer" # noqa S105
def authenticate(self, request):
"""
Authenticate the server-to-server request by validating the Authorization header.
This method checks if the Authorization header is present in the request, ensures it
contains a valid token with the correct format, and verifies the token against the
list of allowed server-to-server tokens. If the header is missing, improperly formatted,
or contains an invalid token, an AuthenticationFailed exception is raised.
Returns:
None: If authentication is successful
(no user is authenticated for server-to-server requests).
Raises:
AuthenticationFailed: If the Authorization header is missing, malformed,
or contains an invalid token.
"""
auth_header = request.headers.get(self.AUTH_HEADER)
if not auth_header:
raise AuthenticationFailed("Authorization header is missing.")
# Validate token format and existence
auth_parts = auth_header.split(" ")
if len(auth_parts) != 2 or auth_parts[0] != self.TOKEN_TYPE:
raise AuthenticationFailed("Invalid authorization header.")
token = auth_parts[1]
if token not in settings.SERVER_TO_SERVER_API_TOKENS:
raise AuthenticationFailed("Invalid server-to-server token.")
# Authentication is successful, but no user is authenticated
def authenticate_header(self, request):
"""Return the WWW-Authenticate header value."""
return f"{self.TOKEN_TYPE} realm='Create document server to server'"

View File

@@ -1,59 +1,122 @@
"""Authentication Backends for the Impress core app."""
import logging
import os
"""Authentication Backends for the People core app."""
from django.conf import settings
from django.core.exceptions import SuspiciousOperation
from django.db import models
from django.utils.translation import gettext_lazy as _
from lasuite.oidc_login.backends import (
OIDCAuthenticationBackend as LaSuiteOIDCAuthenticationBackend,
import requests
from mozilla_django_oidc.auth import (
OIDCAuthenticationBackend as MozillaOIDCAuthenticationBackend,
)
from core.models import DuplicateEmailError
logger = logging.getLogger(__name__)
# Settings renamed warnings
if os.environ.get("USER_OIDC_FIELDS_TO_FULLNAME"):
logger.warning(
"USER_OIDC_FIELDS_TO_FULLNAME has been renamed to "
"OIDC_USERINFO_FULLNAME_FIELDS please update your settings."
)
if os.environ.get("USER_OIDC_FIELD_TO_SHORTNAME"):
logger.warning(
"USER_OIDC_FIELD_TO_SHORTNAME has been renamed to "
"OIDC_USERINFO_SHORTNAME_FIELD please update your settings."
)
from core.models import Identity
class OIDCAuthenticationBackend(LaSuiteOIDCAuthenticationBackend):
class OIDCAuthenticationBackend(MozillaOIDCAuthenticationBackend):
"""Custom OpenID Connect (OIDC) Authentication Backend.
This class overrides the default OIDC Authentication Backend to accommodate differences
in the User and Identity models, and handles signed and/or encrypted UserInfo response.
"""
def get_extra_claims(self, user_info):
"""
Return extra claims from user_info.
def get_userinfo(self, access_token, id_token, payload):
"""Return user details dictionary.
Args:
user_info (dict): The user information dictionary.
Parameters:
- access_token (str): The access token.
- id_token (str): The id token (unused).
- payload (dict): The token payload (unused).
Note: The id_token and payload parameters are unused in this implementation,
but were kept to preserve base method signature.
Note: It handles signed and/or encrypted UserInfo Response. It is required by
Agent Connect, which follows the OIDC standard. It forces us to override the
base method, which deal with 'application/json' response.
Returns:
dict: A dictionary of extra claims.
- dict: User details dictionary obtained from the OpenID Connect user endpoint.
"""
return {
"full_name": self.compute_full_name(user_info),
"short_name": user_info.get(settings.OIDC_USERINFO_SHORTNAME_FIELD),
}
def get_existing_user(self, sub, email):
"""Fetch existing user by sub or email."""
user_response = requests.get(
self.OIDC_OP_USER_ENDPOINT,
headers={"Authorization": f"Bearer {access_token}"},
verify=self.get_settings("OIDC_VERIFY_SSL", True),
timeout=self.get_settings("OIDC_TIMEOUT", None),
proxies=self.get_settings("OIDC_PROXY", None),
)
user_response.raise_for_status()
userinfo = self.verify_token(user_response.text)
return userinfo
try:
return self.UserModel.objects.get_user_by_sub_or_email(sub, email)
except DuplicateEmailError as err:
raise SuspiciousOperation(err.message) from err
def get_or_create_user(self, access_token, id_token, payload):
"""Return a User based on userinfo. Get or create a new user if no user matches the Sub.
Parameters:
- access_token (str): The access token.
- id_token (str): The ID token.
- payload (dict): The user payload.
Returns:
- User: An existing or newly created User instance.
Raises:
- Exception: Raised when user creation is not allowed and no existing user is found.
"""
user_info = self.get_userinfo(access_token, id_token, payload)
# Compute user name from OIDC name fields as defined in settings
names_list = [
user_info[field]
for field in settings.USER_OIDC_FIELDS_TO_NAME
if user_info.get(field)
]
user_info["name"] = " ".join(names_list) or None
sub = user_info.get("sub")
if sub is None:
raise SuspiciousOperation(
_("User info contained no recognizable user identification")
)
user = (
self.UserModel.objects.filter(identities__sub=sub)
.annotate(
identity_email=models.F("identities__email"),
identity_name=models.F("identities__name"),
)
.distinct()
.first()
)
if user:
email = user_info.get("email")
name = user_info.get("name")
if (
email
and email != user.identity_email
or name
and name != user.identity_name
):
Identity.objects.filter(sub=sub).update(email=email, name=name)
elif self.get_settings("OIDC_CREATE_USER", True):
user = self.create_user(user_info)
return user
def create_user(self, claims):
"""Return a newly created User instance."""
sub = claims.get("sub")
if sub is None:
raise SuspiciousOperation(
_("Claims contained no recognizable user identification")
)
user = self.UserModel.objects.create(password="!") # noqa: S106
Identity.objects.create(
user=user, sub=sub, email=claims.get("email"), name=claims.get("name")
)
return user

Some files were not shown because too many files have changed in this diff Show More