Screen-scraping with Javascript & the CSS Object Model (CSSOM)

Vimium, the keyboard browsing extension I work on, needs to figure out which elements are visible – and where they are – in order to label the visible links with navigational hints. I believe the techniques used are equally useful for doing screen-scraping and page analytics via third-party Javascript, so I have documented our findings here.

Determining element position & dimension

The naive way to try and determine an element’s properties is to read off it’s style attribute. However, since the final layout of an element this the result of a complex iteraction between the properties of the element and those of the rest of the document, this is rarely useful. Fortunately, the CSS Object Model (CSSOM) API provides access to the browser’s actual layout values.

For accessing element positions and dimensions, we have two tools at our disposal: getClientRects and getBoundingClientRect. The former returns an array of ClientRects, each with properties left, top, width, and height; the latter is supposed to return the smallest possible box within which all the individual ClientRects fit. left and top are both measured with respect to the viewport. In general, block elements only have one ClientRect, but inline elements can have more, such as when a link is split across two lines. An illustration should make it clear:

Bounding ClientRect for a link split across two lines

Bounding ClientRect for “foo bar” link

Individual ClientRects for a link split across two lines

Individual ClientRects for “foo bar” link

However, even if an element is at a given position within the viewport, it may still be hidden from view. Let us investigate.

Causes of invisibility

There are a variety of CSS styles which could make an element invisible:

These styles apply to both an element and its children, so an element could be invisible by virtue of being instead in another one that has e.g. display:none.

(Note that when we talk about visibility, we are counting an element as “visible” even if it has no text content itself, but contains descendants that have content. I.e. #foo in

<div id='foo'>
    <div>bar</div>
</div>

is visible.)

Detecting invisibility

Once again, looking at the values on the .style field is insufficient to detect invisibility, because our element could be invisible due to a style on one of its ancestors. At first glance, window.getComputedStyle seems like a solution. This function gives us access to the browser’s computed style values. Clearly, this works if the “invisibility” styles have been applied to the current element.

However, of the four styles mentioned, only visibility:hidden gets propagated to all the children of the affected element – i.e. a descendant of an opacity:0 element can still have a non-zero opacity. For an element nested within a parent with display:none, however, there is a workaround: all its children will have no ClientRects, and its bounding ClientRect will have dimensions of size zero. This is nice because it means that checking for zero-sized elements will also catch these elements.

For elements that have been clipped due to overflow, we have to take a longer route using elementFromPoint, which gives us the topmost DOM element at a given screen coordinate. Since we know how to get an element’s position, we can check if the element at that position is one of its descendants. Unfortuantely, this means that we have to traverse the DOM tree in Javascript by looking up parentNode repeatedly, and this gets slow for a large number of elements. Vimium eschews handling this edge case because it is sufficiently rare.

Finally, since opacity:0 elements are still being “rendered” as completely transparent elements, elementFromPoint will still detect them. The only way to deal with this is to walk up the DOM tree to find if a given element has any parents with opacity:0. As before, this is computationally expensive.

Further Edge Cases

Simply having a size zero clientRect does not mean that an element is invisible. Elements which contain only visible floated descendants will also have size zero rects, because floated elements are “taken out of the document flow” and as such do not affect their parents. The only way to check this is by recursing down the document tree.

Finally, we note that WebKit has not implemented getClientRects() correctly for SVG elements. The method returns empty lists for visible SVG elements, and the workaround is to use getBoundingClientRect() instead.

Summary

Below is a table showing what kinds of properties can be detected with the O(1) visibility tests we have described. The results are given for Chrome, Safari, Firefox, and IE9.

The elements highlighted in orange have “unintuitive” results: either they are invisible but pass all our O(1) tests, or they are visible but fail a number of our visibility tests.

The table was generated based off this test page. You may also be interested in Vimium’s code for handling visibility, which is here.

List of Tests

  1. BoundingClientRect has nonzero dimensions
  2. Is visible, displayed, and has opacity > 0
  3. Has ClientRect
  4. ClientRect has nonzero dimensions
Element / Test1 2 3 4
visibility:hidden yes no yes yes
nested in an element with visibility:hidden yes no yes yes
display:none no no no no
nested in an element with display:none no yes no no
opacity:0 yes no yes yes
nested in an element with opacity:0 yes yes yes yes
nested in an element of zero dimensions & overflow:hidden yes yes yes yes
Contains only visible floated elements no yes yes no
Visible, contained within an SVG yes yes Not in Webkit Not in Webkit

Related Posts

Jez Ng 05 November 2012
blog comments powered by Disqus