1

Meeting 2012 07 20

dherman edited this page 2012-07-20 17:13:06 -07:00

Table of Contents

• Attending
• Text
• Rendering
• Floating-point
• Boxes
• Networking
• Content
• Events
• Image decoding

Attending

Boris, Brendan, Dave, Patrick, Niko, Johnny

Text

• Boris: ROC or Jonathan Kew would know more
• Boris: long-term: caching situation needs to be resolved
• Patrick: my instinct: a task manages cache, and everyone queries the task
• Boris: that probably works
• Patrick: depends on speed of message; with Eric Holk's pipes work, I'm a bit more confident
• Boris: textrun process is a pretty noticeable component, timewise; creating textruns, looking them up in cache, etc
• Patrick: I assume we divide into lines; could we spawn a task for every line?
• Boris: textruns span across lines; they can span across a lot of stuff; there are exceptions...
• Boris: part of the problem is when we create a textrun we don't know where line breaks will be; preformatted is an exception, but in general you don't know
• Patrick: you could do it based on blocks maybe
• Niko: that's right, they don't span across block boundaries
• Patrick: that's not an awesome speedup
• Dave: can you do heuristic guessing of where linebreaks will be?
• Boris: we'd have to see what happens when you get it wrong
• Niko: across blocks would get parallel paragraphs? not bad, right?
• Patrick: not bad. it's not the 64cpu future, but it's the 4cpu future
• Boris: long wikipedia-like things; wherever you care about performance you'll probably have lots of blocks
• Boris: exception: one paragraph with tons of text, but that's not so critical

Rendering

• Patrick: I'm working on resizing windows
• Patrick: implemented except for gfx; doesn't have multiple buffer-size support; probably a day's work
• Patrick: we have layers now, we're GL-accelerated now
• Patrick: for rendering: plan to try an experiment: since we do tiling on mobile, I was thinking we could render every tile in parallel; copy display-list for every tile and render them in parallel, ship them to the compositor which glues them together
• Boris: you have to be a little bit careful about seaming
• Patrick: we already solved that on Fennec
• Dave: I still see bugs with seaming
• Patrick: I know what those are, due to subpixel issues with layer invalidation; those won't be a problem in Servo; those are weird subpixel hacks because we can't scroll to device pixel positions; fixing it requires a bunch of nasty stuff in Gecko but we won't have that issue since we'll design it right in the first place
• Patrick: rendering stuff in parallel will be an experiment; don't know how much it'll help, and have to overcome overhead of rendering display list multiple times
• Patrick: basically, doing same operation but with different clip rects
• Patrick: I imagine it'll be a big win for stuff like gradients, where you spend all your time in the guts of pixman doing complex operations
• Boris: question for that is how close we are to moving gradients to the GPU
• Boris: I agree it's worth having the capability, b/c we can always turn it off
• Patrick: we could conceivably have smart things where we layout memory just so, so we can render many tiles sequentially
• Boris: we can play around with it, do some dynamic decisions
• Patrick: tile-by-tile rendering is nice for other reasons: you can ship the tiles to compositor as they complete, display in some cases
• Brendan: that's parallelizable, right?
• Patrick: interpreting the display-list multiple times is overhead though
• Brendan: can you estimate how much that hurts?
• Patrick: not clear yet
• Patrick: or you could ARC the display list, to avoid copying, but you still have to interpret it multiple times, so that's overhead
• Patrick: benwa and I calculated this for Fennec, but the calculus was different because we weren't parallelizing there, just shipping things as soon as they were ready
• Niko: wanna understand: each tile looks at whole list...
• Patrick: doesn't have to do that
• Niko: oh, just executes the whole list. but kinda the same thing
• Patrick: yeah
• Boris: worst case is very big display list where they don't paint very much, so most of your time is processing but not painting
• Patrick: best case is taskjs.org: enormous gradient, cairo/skia killing. we'd probably get huge speedups there
• Boris: doing software gradients is expensive b/c it's per-pixel
• Brendan: why aren't we doing it in HW?
• Boris: working on it
• Patrick: display item is tiny, but work is enormous. so we should get a huge speedup
• Patrick: thinking of asking margaret to do parsing for gradients, since that's easy and could be a good demo. kind of hilarious to do CSS3 before implementing line breaks
• Boris: we should be doing the things that are the hard cases. we should push on the stuff we won't do as well on
• Dave: three kinds of things: things that are boring and uninteresting, things we should do well on, and things that could screw everything up. should be looking at latter for sure. floats, for example?

Floating-point

• Patrick: question for Boris: floating-point in layout?
• Boris: we've had a bunch of annoyance in Gecko dealing with integer overflow
• Patrick: didn't realize that was a big problem
• Boris: we use magic int values to mean "unspecified" or "unconstrained". just magic int values! nothing to prevent CSS from using those values, which hits assertions...
• Boris: fair amount of annoyance there. not necessarily in terms of failing on real-life web pages, but fuzz testing and what not. difficult to assert your invariants when they don't hold.
• Patrick: you can encode sentinels easily with floats with NaN boxing
• Boris: and you can use Infinity for unconstrained, etc. so that's a plus.
• Boris: that's IMO the main plus. that and we to some extent get sub-pixel layout "for free".
• Boris: third plus: no weirdness where depending on how you specify your style you get zero or not. right now our computed style rep is quantized. can cause problems in some situations.
• Boris: bad things: performance is a concern. especially on ARM though less so now since it has HW floating point. rounding error is a concern for two reasons: 1) parts of CSS like margin collapsing that depends on whether edges coincide. maybe we can do it in other terms, but we'd need to be careful to deal with rounding error one way or another.
• Brendan: epsilon testing?
• Boris: CSS says one of these things can be offset by less than epsilon
• Boris: but maybe we can do this some other way than FP. ROC would know.
• Boris: 2) say you have a block w/ bunch of kids. one kids has huge height, and a bunch of kids with 2-pixel heights. trying to compute height of parents. sum height of kids. order of operations matters for precision.
• Dave: is this something where size of screen can create limits?
• Boris: no, b/c page can be unbounded size. possible to create infinite page.
• Patrick: example: ebook novel
• Boris: not sure I've said anything you guys haven't thought of. answer is I don't know.
• Boris: worst case scenario maybe we switch from float to double and that mitigates rounding error issues
• Boris: in Gecko in theory we have ability to #ifdef nscoord. but in practice it's difficult to code to that b/c you need to be very careful what kind of operation you're doing
• Dave: we could possibly do some tricks with type classes
• Boris: that's an option, maybe that's the way to go. instead of doing arithmetic, sometimes use different higher-level operations
• Dave: sounds like premature abstraction, if it's just b/c we don't know which rep we need
• Boris: ok, if it's hard to write, we shouldn't do it.
• Boris: my feeling overall is we have some intuitions and experience already with ints. my fear is float is second system syndrome. "let's do this other thing for which we don't know the drawbacks." but I'm interested in trying floats and see where it goes
• Boris: if we do that, we should try to think of a set of worst-case scenarios and see how they work. if we handle those ok, then life is good

Boxes

• Patrick: ROC was talking about how we do this, and I wasn't entirely sure what he meant. seemed like mostly he was talking about, for line breaks, don't treat them as multiple box objects, just one box object with a line-break structure that indicates where the breaks are happening
• Boris: that's what he's suggesting, yes
• Patrick: he framed it in overarching terms but it sounds like he was really just talking about inline layout
• Boris: yes, inline layout specifically. fundamentally inline layout works in flattened list of things. way Gecko works now: box structure converted to list, do inline layout on list, then go back to boxes. kind of annoying
• Boris: other problem we have: if spans nested several levels deep broken across lines, O(depth nesting * number of boxes) -- noticeable memory & perf cost
• Patrick: when we see inline flow, ... so what do we call these things if "box" doesn't mean "box" anymore
• Boris: in gecko straightforward. but diff representations for diff kinds of floats will need some sort of better API
• Patrick: I like ROC's idea of virtual box iterator: "given this thing, tell me what the boxes are" -- creates those on the fly
• Boris: yeah
• Patrick: so question is what to call these. webkit calls them "renderobjects" or something. don't like that name
• Boris: so what are these things really...
• Patrick: "flow"?
• Boris: question is whether we have a single name for things in inline vs block layout
• Boris: element renderer? not quite b/c we want them for text nodes too
• Niko: I'm confused. we have dom tree, talking about for inline layout, rather than creating boxes that appear in the CSS spec...
• Patrick: they don't mirror dom; they're the flows. there will be one "renderer object" which represents an inline flow. all of spans point to this one object
• Boris: interesting. still don't know quite how to set this up
• Boris: was thinking one render object per span
• Patrick: so they get joined together into an inline flow?
• Boris: don't know. maybe right thing is single object. issue is you need to iterate boxes corresponding to single span
• Patrick: probably pointer into inline flow. "here's where in the flow this span is". way boxes done is per-line
• Patrick: you have to have some .eachBox or something
• Niko: so inline flow is like a list of all of the leaves of this tree, with some additional data structures with the tree structure. span knows subset of list that is its content
• Niko: inline flow also has points in that where the breaks occur
• Patrick: yeah. so two sets of pointers into list. elements pointer which comes from dom nodes, and line breaks which comes from layout
• Boris: we can turn things like padding and borders into the list as well. have to be careful where to put them in list b/c of line breaks
• Patrick: sounds cool
• Niko: seems logical
• Patrick: question is what are they called
• Niko: inline flow seems good
• Patrick: kinda like flow
• Boris: inline flow works for me
• Patrick: and block flows
• Niko: block flow is same as CSS box
• Boris: until you start doing columns or pagination
• Boris: some similarities to inline flow, but algorithms are different
• Patrick: but both can expose .eachBox
• Boris: yep

Networking

• Patrick: working on network, stuff like caching will come up. right now Brian's hooking up bindings to joyent's HTTP parser
• Boris: whatever we do, we're gonna need a real networking lib
• Dave: Jason does want to work on this for us, just not yet
• Boris: we can use whatever for now
• Patrick: we can use libuv in production
• Boris: sure, basic sockets is fine
• Patrick: they handle annoying Windows IOCP vs Unix kqueue
• Boris: plan for SSL? NSS?
• Johnny: yes
• Boris: we'll need a way of exposing libuv sockets to NSS
• Boris: NSS is NSPR happy. but we should avoid NSPR if we can
• Patrick: what does Node use?
• Johnny: probably OpenSSL
• Boris: two plausible alternatives: NSS, OpenSSL. both have serious drawbacks
• Patrick: assuming NSS FIPS-ceritifed.
• Boris: particular version we're shipping may or may not
• Brendan: let's talk about NSS. chrome using it too and we're still investing in it. don't see why we'd switch
• Boris: NSS assumes sockets are NSPR and stuff. we'd need to somehow wrap whatever we're doing in NSPR abstraction
• Dave: additional abstraction cost?
• Patrick: maybe. uv sockets + NSPR abstraction
• Boris: thing to look at: from Necko's point of view, it's a socket provider. we should look into what NSS depends on, maybe talk to bsmith. maybe it's just not an issue and we can just make it work.
• Johnny: does it really matter? if we can just use NSPR, do we need libuv?
• Patrick: yes, b/c of the way the scheduler works.
• Patrick: well, can Necko do async on all platforms?
• Johnny: yes.
• Boris: meaning what exactly?
• Patrick: basically every operation takes callback
• Niko: nonblocking I/O
• Boris: in theory Necko set up that way; in practice front-end not thread-safe. underlying code is thread-safe
• Patrick: other problem is XPCOM
• Boris: NSPR doesn't involve XPCOM
• Boris: NSPR will be needed for NSS no matter what, but whether we use it for sockets is the question
• Patrick: would like to use GLUT to add a little "right-click and enter URL" UI

Content

• Patrick: once we have networking + text, we'll have the entire pipeline of a browser
• Brendan: except for JS lexing + parsing, HTML lexing + parsing
• Patrick: jdm has document object working in JS; Node object is next, working on that
• Patrick: hoping that stuff can land soon
• Patrick: once we have that we'll have all the components of a browser in minimal form
• Johnny: there's stuff like script loading & execution
• Patrick: that's working, isn't it?
• Niko: no, doesn't take scripts out of web page
• Boris: can we block parser?
• Niko: html parser?
• Patrick: don't think so, no
• Boris: that we'll need to change
• Johnny: at some point we'll use real HTML parser
• Niko: there's a very small Rust bug blocking that. should prioritize
• Patrick: yeah, labelled break + continue
• Dave: have to touch base with Zhijia about his project
• Boris: once you hit a script, can inject random stuff into character stream
• Brendan: yeah, scripts are evil without async
• Niko: how common to inject?
• Boris: common, but scripts do crazy things like unbalanced comments piece at a time
• Boris: we have to be super careful if we aren't following the state machine
• Johnny: I'd say HTML parsing perf is rarely a bottleneck
• Brendan: Amdahl: could end up a bottleneck if other stuff gets fast
• Boris: can do tests
• Niko: even stuff like emitting a comment start & end tag is not that bad. emitting closing > sign is really bad
• Boris: could emit opening < sgn
• Niko: that kinda stuff we're just screwed on
• Dave: it's ok to do badly on these cases
• Patrick: I'd rather focus on other things than parallelizing parsing
• Boris: it's isolated in its own component
• Patrick: saying it's sequential now doesn't preclude parallel later
• Dave: and we're just starting with Zhijia

Events

• Patrick: just started adding window resize events. goes to content task, not yet to JS
• Patrick: so... yay

Image decoding

• Brendan: Company100 has been working on parallel image decoding for common formats; I'll look at that, Rust, image decoders
• Brendan: we did this in the 90's; farm out macro blocks
• Patrick: writing that in Rust wouldn't be that hard
• Brendan: besides harfbuzz, there was a PNG exploit the other year, so this stuff is valuable to move to Rust
• Boris: BMP bug filed the other day: heights can be negative meaning come up from the bottom not from the top
• Brendan: and favicons are BMP