Why is it crucial that you recognize what is in our web archives? Probably the main reason is the fact that as an endless and ever changing landscape, it’s just impossible to archive the “whole net” and absolutely maintain every change to every page in existence. Web archives are building them is an exercise in innumerable tradeoffs of the best way to sustain an endless stream with limited resources and by their very nature an imperfect record of the internet.

At the most fundamental level there’s the question of precisely what types of sites they need to prioritize and the best way to seed the crawlers of an archive. Should an archive prioritize authorities content as a lens on the tips a government provides its citizens, educational sites as the end product of the state’s centres of education, commercial sites as an indication of commercial utilization of the internet or private sites as a window onto civil society?

To what degree should an archive effort to maintain expertise and the content of dynamic sites including personalized and interactional sites or databases and search engines?

There’s no single “right” response to some of these questions. The paper section of a national library may be interested in maintaining a minumum of one copy of every article published through an internet news website in the nation. A political communications scholar, on the flip side, might need to monitor how authorities press releases are being changed over time or the development of a leading political blog over many years. The prior allocates all action that is crawling to locating links that are new, while the latter demands exact constant high-density photos over decades.