Meeting with build reps on getting the Rust compiler in Gecko

Like to understand what the short-term vs. long-term requirements are
e.g., is there a way to stage through "just in nightly" to "just in beta" to "OSX only release" before supporting the full matrix?

Overview

larsberg: We'd just like to ship something in 2015. One platform is fine.
ted: need a mozbuild file. You will probably want to expose a C API, more than likely. Likely a standalone, shared library the libxul links again? Can we compile Rust libraries into libxul and link them together?
acrichto: Yes.
ted: Then just need build to know how to build them?
glandium: Yes.
gps: Anything it pulls in? libllvm?
brson: Self-contained.
glandium: jemalloc?
acrichto: When you compile rust, you can enable or disable it. I suspected you'd want to disable it.
ted: Can you use ours?
brson: Yes.
gps: Anything special on windows?
brson: We use mingw right now.
gps: Visual Studio? Is it a goal?
brson: Ambition...
ted: We build with Visual Studio on Windows. Not negotiable. Does clang-cl solve this?
ehsan: Yes.
build: Probably may be solved for you, if you shift. Talk with ehsan. Would be OK not to support Windows in the first pass as well. Maybe debug symbols will be funny?
gps: Either way, that's a toolchain dependency to windows that is new.
brson: WE can provide the toolchain.
nfroyd: Only need mingw when compiling, right?
brson: There are components in the Rust toolchain that require mingw.
glandium: If the rust compiler is self-contained, that's great.

Deploying to build machines

build: Use Tooltool. It's a JSON file with SHA hashes for tool packages. You provide a tar file of that tool, it gets stuck in a approved location, and the build machines pull it down as part of the build.
gps: tooltool is not enabled on all builders. Just Mac...
glandium: Everything that matters. Not the Valgrind build. Not on the SM build.
ted: Have to figure out valgrind. Valgrind doesn't use mozharness.

What version of Rust?

ted: Provide a new tarball and update the manifest. Not a big deal. Don't want to do it constantly. Plus, your dependencies become those of build. If you have a lot of churn, you will make developers mad.
jack: One every six weeks a lot?
nfroyd: Yes.
gps: Is tooltool public next year?
ehsan: Almost.
ted: In terms of non-releng, we have bootstrap scripts in the tree. Can apt-get, brew install makes it great. Windows is more of a hassle, so we just have a monolithic mozilla build package, but if you have to update it, that's hard.

Distro support

gps: distro packaging.
ted: If we can keep it optional for a while, we can handwave for a while.
gps: Buys time, but not forever.
jack: Once rust gets into RC phase, it will get packaged.
ted: If you need something newer, that's OK. Have to do it for clang drops. Imposing it on other developers is harder. We don't change our toolchain that often.
gps: If it's optional it satisifies most of our complaints. But it'd be nice replicate the release build as much as possible to avoid the try build.

Compiler efficiency

gps: Dependencies. How efficient is the compiler about no-op'ing if it needs to. How much time does rebuilding take. WE're finally down to incremental FF builds in seconds.
nalexander: is rustc compilation-unit c like or java-like.
mbrubeck: One library is one compilation unit.
nalexander; So it's java-like. So gps's question is relevant.
nalexander: can rustc emit a .ppdeps file?
jack: Yes, rustc can emit it.
gps: That is what we expect it to work.
jack: Maybe cargo?
larsberg: Let's just have mozbuild invoke rustc, works like normal makefiles.
ted: Yes.
gps: Better would be if somebody can sit down with mike shaw (of tup).
brson: Build times.
gps: Twitter failed with Scala's inability to scale to incremental linking. A single input change would result in 30 seconds.
jack: It's like that in Rust, but it's to the crate-level. We try to keep our crates really small. Servo itself, because it has a bunch of code, can be slow.
brson: If your dependency graph hits one big create, we have issues.
gps: The concern is that if we end up with a 30 second link time, we'll have a problem.
brson: It would help us to prioritize our 2015 planning if we know what your requirements are.
larsberg: We won't see it much now because we will replace small libraries first.
Q: We're pushing to change the Android / Java-side build,w hich has the same problems that Rust has (your crate is our jar). We're also pushing to an external build system that looks more like cargo - move to gradle and get out of the business of building Android things. WE ahve the same concerns. We'll just invoke it instead of javac. Putting that money into Cargo long-term would be good. That's what's happened with our Android builds. I'd push strongly for that.
nbp: I totally disagree with language package managers, especially ones that fetch packages while building.
ted: It's different if Cargo is going online - that's a no-go.
nalexander: Agreed; that's one of my biggest concerns about gradle.
jack: There's no magic in Cargo; it just calls rustc under the hood anyway. Could also have cargo emit the depinfo, if we want to try that instead.
ted: If we're not building a standalone rust thing and link it into libxul, does that work with Cargo?
acrichto: Yes.

Artifact caching

gps: Artifact caching. Knowing all of the inputs into what you're compiling and the outputs so you can grab it from S3 - ccache for C/C++. We use it on linux, mac, and windows. So, what do we do to cache Rust outputs? It's sped up the builds.
brson: Hard requirement?
gps: It will be an issue as the number of Rust components increase. Build needs to be deterministic.
jack: I believe it's been a cargo goal from day 1.

Overhead of producing rust binary

gps: Small component that's a few hundred lines. How big is the library?
acrichto: Our static library is included, so there's a baseline. We should work with LTO.
larsberg: What is the smallest, total.
acrichto: 600kb. No debuginfo.

Symbol info

nbp: Gdb support?
acrichto: Well. DWARF output. All should be integrated.
brson: We do masquerade as C, but not everything you can do in gdb works. You can evaluate C. Unfortunately, Windows is also in bad shape.
ted: Will clang on windows do PDBs?
jrmuziel: line number info and basic stack, but no frame pointer unwinding right now.
eshan: Can emit dwarf and use on Windows...
ted: Problem here is Windows Firefox is VC++, and linking in Rust will not make it possible to debug as a whole.
gps: Ties into crash reporting, because it relies on symbol dumping.
jrmuziel: What breakpad needs is a way to unwind the stack, emit the information that VS does on Windows, so that you can unwind.
ted: Function symbols are a problem... does the linker take care of it?
jrmuziel: Linker will turn it into a PDB.
ted: I think that crash reporting will work and we'll get line info out of it. Debugging should work after a fashion. Probably see some basic source linking; just no locals.
jrmuziel: Adding support for emitting debug information is tractible. Google is not going to do that; they're planning to use lldb when using clang-cl.
ted: Normally it's one or the other; here we're trying to merge.
eshan: We're interested in making this work in clang-cl.
jack: Why are we doing clang-cl, btw?
jrmuziel: Not supporting being built with a compiler we don't have source to.
gps: That's a much later discussion.

PGO

gps: PGO issues with Rust? Nah...

How do we ship a feature?

ted: Once it's enabled in nightly as a build, just have to not be causing a regression.
eshan: Have to get
nalexander: Test infrastructure for rust?
acrichto: Run the compiler with --test and you get a binary.
gps: That's an automation issue.
glandium: Also depends on the component. If it's something exposed to teh web, you need itool or web tests, too.
ted: ICO, BMP decoder, etc. very easy.
glandium: If it replaces something in Gecko that has tests, we have to pass them.
ted: Do we have ImageLib tests? If so, we should just get them and have to pass them.

Different platforms

larsbeg: Can we just ship on Linux or OSX first?
ted: It's OK.
eshan: Can't make a general statement.
gps: Have to talk with the module owner. There should be a path to it on every platform.
ted: Can also say one platform in nightly, but not in aurora until on all platforms.
jack: Is windows the only long pole?
gps: Usually.

Getting into Gecko build

eshan: Timeframe?
larsberg: Across 2015. By end of year, something that ships.
gps: From a build system perspective, if it's all behind a flag, you can do that right now. It's just when you change things that ship in Firefox that you have to talk to people, outside of the build team.

Toolchain

jrmuziel: What's the size?
acrichto: 110MB download.
gps: Same as GCC.
nalexander: Source or the binary?
ted: That's gonna suck for people in developing nations.

Other platforms

larsberg: Solaris?
glandium: x86 solaris.
kmc: LLVM is dropping compiling code supporting Windows XP.
eshan: Dropping support for building on XP, not for it.
larsberg: rust does not support targeting XP.
brson: No major reason.
acrichto: if you use condvars, you can't target it.
jrmuziel: That's an issue.
eshan: We have our own condvar. Support Windows XPSP2.
ted: Not Solaris.
glandium: If you leave the Gecko code behind, it's OK to leave the tier 2 platforms.
brson: We need to bootstrap up FreeBSD, too.
jrmuziel: SPARC LLVM backend?
ted: Nope.
brson: MIPS? Is it required?
ted: Not really.
gps: ARMv6 and ARMv7.
glandium: And ARMv5t.
jack: Plan for ARM64?
nbp: We have will have it by end of December.
nfroyd: Geck already runs on ARM64 w/o the SM JIT.
ted: Don't stress about tier 2. Win32. Focus on 64/32 Linux, 64/32 OSX, 64 Windows, ARM/Android. ARMv6 and ARMv7.

Advice / next steps

gps: Send a post with the plan to dev-platform to get more feedback.
ted: Also, open bugs.

XPSP2

brson: How long will you support it?
eshan: Once we stop having users. We still had millions of XPSP2 users.
gps: Around 10%.
glandium: There are more XPSP2 than Linux users.

OSX

brson: How far back?
gps: 10.6. Haven't seen numbers recently, though.
glandium: Still there.
frodynj: high single digit percentage.
gps: bsmedberg said we still have to build 32-bit gecko until netflix departs silverlight and goes to EME.

Linux Kernel

brson: Minimum kernel version.
eshan: 2.4
acrichto: if I said 2.6.18...
eshan: Fine.