112 Commits

Author	SHA1	Message	Date
Alezander9	61f9c31a3d	feat: support changing eval task set	2025-06-03 10:27:34 -07:00
BroskyBrowser	15cb992618	fix: expose ANCHOR_BROWSER_API_KEY variable in the evals workflow .yaml	2025-06-01 12:58:14 +02:00
Nick Sweeting	bfb6b26274	Merge branch 'main' into new-eval	2025-05-25 18:58:08 -07:00
Nick Sweeting	1d2cb46d73	Update claude.yml	2025-05-25 06:07:37 -04:00
Nick Sweeting	6d0758764a	Add Claude PR Assistant workflow	2025-05-25 03:04:06 -07:00
Alezander9	ad71ba8d29	add branch name selection into workflow	2025-05-24 19:12:01 -07:00
Alezander9	1f113fa640	Merge remote-tracking branch 'upstream/main' into new-eval	2025-05-24 10:25:43 -07:00
Nick Sweeting	04a3c881df	add docker setting for ci	2025-05-23 22:22:10 -07:00
Nick Sweeting	cacc7c2020	allow running publish manually	2025-05-23 22:18:09 -07:00
Nick Sweeting	8e8f9a2381	allow running publish manually	2025-05-23 22:17:09 -07:00
Nick Sweeting	4196e79faa	fix publish action	2025-05-23 22:15:34 -07:00
Nick Sweeting	3d10260543	fix missing link between find_tests and test job in CI	2025-05-23 19:22:55 -07:00
Nick Sweeting	37a36dbd28	catch failure case up-front	2025-05-23 19:12:46 -07:00
Nick Sweeting	6a1ed628e3	properly split filenamees out of ls results in test discovery	2025-05-23 19:10:25 -07:00
Nick Sweeting	063f103efd	more warning on filure to list tests	2025-05-23 19:05:52 -07:00
Nick Sweeting	06ee004a88	add assertion to tests discovery	2025-05-23 18:33:18 -07:00
Nick Sweeting	9fcd5cd7b2	debugging tests discovery	2025-05-23 18:32:28 -07:00
Nick Sweeting	e19e1c5dfc	fix ci tests	2025-05-23 18:29:59 -07:00
Nick Sweeting	815ae48938	fix pypi publish action	2025-05-23 18:21:18 -07:00
Alezander9	45dd0a26c2	update eval workflow with new arguments	2025-05-23 14:57:18 -07:00
Alezander9	a3dd8b004b	update eval workflow with new arguments	2025-05-23 14:46:14 -07:00
Nick Sweeting	27ca169393	reduce mumber of redundant CI builds	2025-05-23 04:46:21 -04:00
Nick Sweeting	3940462d8d	Potential fix for code scanning alert no. 28: Workflow does not contain permissions ... Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>	2025-05-23 03:50:15 -04:00
Nick Sweeting	6b8360c475	better logging	2025-05-22 23:17:21 -07:00
Alezander9	aab470243f	update user message default argument	2025-05-22 15:53:49 -07:00
Alezander9	529e43fdd1	update args in workflow script to match new format	2025-05-22 11:36:47 -07:00
Alezander9	4a7e9113ca	add claude 4 support and cleanup eval script arguments	2025-05-22 10:54:00 -07:00
Nick Sweeting	18554e2834	autodetect tests for ci by looking in folder	2025-05-22 05:34:04 -07:00
Alezander9	0fbfc82da0	switch eval workflow to use new consolidated branch	2025-05-21 14:39:09 -07:00
Alezander9	a8d661b2d0	consolidated changes: adapt refactored eval service to work with new browser and on github actions	2025-05-21 14:36:57 -07:00
Nick Sweeting	bdb9bc81a3	install both chrome and chromium channels	2025-05-20 03:55:57 -07:00
Nick Sweeting	312a738ce9	better browser info logging at startup and tests	2025-05-20 03:50:04 -07:00
Nick Sweeting	836e1ddbf0	rename test	2025-05-20 02:33:33 -07:00
Nick Sweeting	eb8e7d52e0	only run docker build on main branch pushes	2025-05-16 05:42:58 -04:00
Nick Sweeting	71329da1d5	use repo name for ghcr	2025-05-13 22:09:46 -07:00
Nick Sweeting	56a9ed7374	more dockerfile fixes	2025-05-13 21:24:32 -07:00
Nick Sweeting	a771816566	also push to ghcr	2025-05-13 21:16:42 -07:00
Nick Sweeting	150dc9efde	also push to ghcr	2025-05-13 20:34:36 -07:00
Nick Sweeting	ac08799202	improve docker autobuild tagging	2025-05-13 20:11:45 -07:00
Nick Sweeting	5dc7a3cee6	feat: add dockerhub workflow (#1608)	2025-05-13 20:10:36 -07:00
Nick Sweeting	7e26eb14b1	add glob support to allowed_domains	2025-05-13 18:25:28 -07:00
Nick Sweeting	3f4c918acf	fix tests to use playwright too	2025-05-09 18:22:29 -07:00
danilaplee	4a1111c604	feat: add dockerhub workflow	2025-05-07 20:54:30 +02:00
Nick Sweeting	4f625fd762	nevermind we still need uv run	2025-05-06 19:03:22 +08:00
Nick Sweeting	38dfb8e36e	rely on already activated venv	2025-05-06 19:01:25 +08:00
Nick Sweeting	005a1310bb	see if tests work without fonts for speed	2025-05-06 18:58:42 +08:00
Nick Sweeting	de7b4e1c82	install pkg deps separately from playwright	2025-05-06 18:13:46 +08:00
Nick Sweeting	aa26ac1850	jk	2025-05-06 17:57:42 +08:00
Nick Sweeting	0164f8d9e7	only install CI-ready version of chromium	2025-05-06 17:57:06 +08:00
Nick Sweeting	062654e532	fix github actions CI tests	2025-05-06 17:54:37 +08:00