RoadMap Oct 2020

State

As of October 2020, development has reached the last big milestone (WEB UI rewrite).

After that, there will be a 'test && polish period', addressing issues in the POST section (plus selected others where appropriate).

Milestones

  • New deb package layout: mini-buildd (the service), mini-buildd-utils (for users), python3-mini-buildd, mini-buildd-doc
  • Code migration to python 3.
  • Update to django 2.2 (minimal version).
  • Switch to twisted as http backend.
  • Support secure data channels (HTTPS && SFTP).
  • API upgrade to json.
  • Event message queue.
  • Periodic task scheduler (cron, internal use).
  • Automated repository snasphots (weekly, monthly, yearly).
  • WEB UI rewrite.

2.0 will

  • be 'upgrade-compatible' with 1.0 (especially, there is no SQL scheme change).
  • not be run-time compatible with 1.0.

Issues

Most of these detailed "issues" are in a 'note to self' fashion (possibly hard to understand for outsiders).

Done issues will be removed from this list (consult the Debian changelog for done things).

WEB UI rewrite

  • General clean up: CSS and templates.
  • Harden against "dict has changed size during iteration" error: Select context, copy context.
  • js, copy2clipboard
  • API: Keep argument values per session?
  • NEW/buster: django source order
  • [WISH]: Web: Support per user or client time zone.
  • [CHECK] http://tools.ietf.org/html/rfc5147
    • Link to parts of the build log via fragment to text/plain

BUILDING && REPOSITORY

  • [WISH] Library of predefined sbuild --chroot-setup-command snippets (eatmydata, ccache)
  • QA
    • autopkgtest support?
    • reproduciblity
    • piuparts support?
  • DEB_BUILD_OPTIONS vs DEB_BUILD_PROFILES
  • [BUG]: When service is stopped, sbuild continues (try w/ chromium, it builds in 13h).
  • [WISH] rm libeatmydata as default (?).
  • [BUG]: When source switches to another archive, you can update the chroot via chroot.check(). However, this needs to be done manually (confusing). Maybe this can be done prior to building?
  • [WISH] Better Daemon defaults for
    • Build queue size
    • Sbuild jobs: Is it safe to default to -j auto or -j N with N > 1 ???
      • debhelper-based packages only actually use this if enabled explicitely ('dh --parallel')
      • ?? Any cases where this could cause parallel builds where not wanted?
  • [CHECK]: Can we use more finegrainded PRIOS on archives (instead of NotAutomatic, ButAutomo...)??
  • [BISH]: auto-deactivation:
    1. When repo is (auto-)deactivated, API calls like show and search don't work any more either.
    2. When one source is auto-deactivated (likely for rolling unstable+testing), the whole repository is deactivated.
  • Sources/Archives: Support partial arch archives (?)
  • [WISH]: Timeout when checking sources (?)
  • [WISH] Make passive ftp port range configurable.
  • [FISH] Make use of [python3-]distro-info
  • [FEAT]: Implement "wake remote" support.
  • [FEAT]: config: Add "purge" action (bulk-delete instances) for chroots.
  • notify: Admin notify on deactivated daemon.
    • ->Daemon.notify for admin messages, repo.notify for packages.
    • ->Daemon.notify for email, rest as before

BUGS

  • [BUG] model data: need restart?
  • [BUG] Intermediate unpredicatble state of daemon instance:
    After Daemon instance creation and before any initial _update_from_model() call, the singleton daemon instance is in an unexpected state for some code (value set to None in constructor). This may lead to confusing errors. Reproduce: './devel update' with the sleep set to 1 seconds.

REVISIT/POST

  • CANNOT REPRODUCE: LOG: daemon.log (file log only) stops working after some time???
  • CANNOT REPRODUCE: events: deadlock just calling events several times(?).
  • CANNOT REPRODUCE: on portext: changes.gen_buildrequest: UnboundLocalError 'mini_buildd' on 'if f["md5sum"] == mini_buildd.misc.md5_of_file(p):'
  • TEST mini-buildd-ssh-*-command setup.
  • debconf (as late as possible, merge local branch).
    • debconf, auto SSL?
  • Documentation
    • DOC: create Debian NEWS file from doc/NEWS
    • [DOC] "../downgrade.buster-ab.pref" (if changing to '~9').
    • [DOC] Quickstart, APT bootstrap: Test and check for jessie, stretch, buster.
  • [BUG] twisted in unstable 18.9.0-11 does not work for events impl (?). 20.3 (exp) ok, 18.9 (buster) ok.
  • CANT REPRODUCE: gpg-agent left-overs (?)
  • CANT REPRODUCE: sbuild_kill, package building keeps running
  • Keep status of packages/builds between shutdowns
    • sbuild_kill: Leaves chroot snasphot?
  • twisted warnings??
    • Sometimes on shutdown: stacktrace: Request.finish called on a request after its connection was lost; use Request.notifyFinish to keep track of this.
    • SSL: 2020-06-29 15:24:11,220 W: /usr/lib/python3/dist-packages/twisted/internet/_sslverify.py:238: DeprecationWarning: NPN is deprecated. Protocols should switch to using ALPN.
  • [CHECK] (unverified) How to change user upload permissions?
  • salsa:
    • ftps patches for dput/dput-ng.
    • gbp: allow setting --distribution for --snapshot.

Post 2.0

  • External user management (ldap, sso?)
  • gnupg: Switch to 'python3-gpg' (NOT 'python3-gpgme').
  • [WISH]: Auto ports and/or Port-Alarm on Debian Package Updates
  • SQL schema migration (with django >= 1.7)
  • Extended User Role Concept, with advanced right managment workflows.
  • [FEAT]: Add suite "wildwest", including some way to add pre-build packages.
  • API: debdiff
  • setup:
    • sync config between instances
    • sync config w/ defaults
    • [WISH] admin/model objects: Show "used by count" (so we can see what objects may be obsolete).
    • [WISH] Archive: Rethink Archive::mbd_ping (see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=740576)
    • [WISH] Integrate ssh upload (using mini-buildd user?) (currently in examples only)
  • [WISH] Automatic SSH upload support, either by scripting the setup or integrating SSH w/ twisted.
  • REPOSITORY
    • sid-test-nakedgoose: Add extra suite for arbitrary package (deb+src) installs.