Spatial data sharing is increasing commonplace and facilitated by database and networking technology. Also growing in availability and importance is presentation of spatial data as part of virtual realities, allowing people to see the data in the context of its recognizable environment. This paper describes important preliminary steps in bringing these trends together to add carefully developed and managed object and image libraries to our spatial data infrastructure. Such libraries will facilitate automated creation of virtual worlds from our digital data stores. The collections will include not only specific buildings, such as seen in Google Earth, but also collections of fully textured, buildings that are typical to specific geographical, cultural or socio-economic regions; trees, shrubs, crops, weeds, and domestic, native and feral animals which represent specific, mapped, ecological or agricultural classes; cars, trucks, street furniture and so forth. In many cases these objects will be full three-dimensional models sometime complete with rules of animation. Much of the vegetation, on the other hand, may be stored simply as images to be used as billboards in the virtual worlds. Such a library requires multiple images of each tree species, in different stages of growth and health, for complete coverage. While object warehouses exist on-line already our envisioned library seeks to be comprehensive, involves multiple organisations and recognises that object have spatial dependencies. The paper reviews the exciting challenges ahead, suggests some solutions and describes potential use cases.