Category Archives: Technical

archive.org download counts of collections of items updates and fixes

Posted on January 26, 2015 by traceypooh

Every month, we look over the total download counts for all public items at archive.org. ?We sum item counts into their collections. ?At year end 2014, we found various source reliability issues, as well as overcounting for “top collections” and … Continue reading →

Posted in Audio Archive, Books Archive, Education Archive, Image Archive, Live Music Archive, Movie Archive, Music, Software Archive, Technical, Video Archive | Tagged , , , | Comments Off on archive.org download counts of collections of items updates and fixes

Using Docker to Encapsulate Complicated Program is Successful

Posted on November 14, 2014 by brewster

The Internet Archive has been using docker in a useful way that is a bit out of the mainstream:?to package a command-line binary and its dependencies so we can deploy it on a cluster and use it in the same … Continue reading →

Posted in Music, Technical | 3 Comments

Job Posting: Web Application/Software Developer for Archive-It

Posted on August 6, 2013 by internetarchive

The Internet Archive is looking for a smart, collaborative and resourceful engineer to lead and do the development of the next generation of the Archive-It service, a web based application used by libraries and archives around the world. The Internet … Continue reading →

Posted in Archive-It, Jobs, News, Technical | Comments Off on Job Posting: Web Application/Software Developer for Archive-It

How to use the Virtual Machine for Researchers

Posted on July 4, 2013 by brewster

Some researchers that are working with the Internet Archive, such as those at University of Massachusetts, have wanted closer access to some of our collections. We are learning how to support this type of “on-campus” use of the collections. This … Continue reading →

Posted in Technical | Tagged | 2 Comments

The Internet Archive Metadata API

Posted on July 4, 2013 by internetarchive

The Metadata API is intended for fast, flexible, and reliable reading and writing of Internet Archive items. Metadata Read API The Metadata Read API is the fastest and most flexible way to retrieve metadata for items on archive.org. We’ve seen … Continue reading →

Posted in Technical | Tagged , , | 1 Comment

Site down some of Tuesday and Wednesday for Power Upgrade

Posted on April 15, 2013 by Ralf Muehlen

[Update:?? Upgrade is done, we were offline twice, as we predicted (and are sorry about), but now we have twice the power. Thank you PG&E, Ralf Muehlen, and the Archive engineers.] This week, we are doubling the power coming into … Continue reading →

Posted in News, Technical | 3 Comments

Celebrating 100 million tasks (uploading and modifying archive.org content)

Posted on April 12, 2013 by traceypooh

Just over 8-1/2 years ago, I wrote a multi-process daemon in PHP that we refer to as “catalogd”. ?It runs 24 hours a day, 7 days a week, no rest! It is in charge of uploading all content to our … Continue reading →

Posted in News, Technical | Tagged , , , , | 1 Comment

new mp4 (h.264) derivative technique — simpler and easy!

Posted on February 9, 2013 by traceypooh

Greetings video geeks! ??? We’ve updated the process and way we create our .mp4 files that are shown on video pages on archive.org It’s a much cleaner/clearer process, namely: We opted to ditch ffpreset files in favor of command-line argument … Continue reading →

Posted in Technical, Television Archive, Video Archive | Tagged , , , , , | 3 Comments

new video and audio player — video multiple qualities, related videos, and more!

Posted on February 9, 2013 by traceypooh

Many of you have already noticed that since the New Year, we have migrated our new “beta” player to be the primary/default player, then to be the only player. We are excited about this new player! It features the very … Continue reading →

Posted in Audio Archive, Live Music Archive, Technical, Television Archive, Video Archive | Tagged , , , , | 4 Comments

files.xml and metadata API now showing video file’s width, height, and duration

Posted on February 9, 2013 by traceypooh

For those video oriented of you who inspect our _files.xml directly, or use a metadata API (JSON) for items, you’ll likely be pleased to know that for all new items, as well as items that update (eg: review posted, metadata … Continue reading →

Posted in Technical, Television Archive, Video Archive | Tagged , , , , | Comments Off on files.xml and metadata API now showing video file’s width, height, and duration

getting only certain formats in .zip files from items — new feature

Posted on October 30, 2012 by traceypooh

Per some requests from our friends in the Live Music Archive community… You can get any archive.org item downloaded to your local machine as a .zip file (that we’ve been doing for 5+ years!) But whereas before it would be … Continue reading →

Posted in Audio Archive, Live Music Archive, Technical | Tagged , , , | Comments Off on getting only certain formats in .zip files from items — new feature

Uploading images for text items (update on *_images.zip format)

Posted on May 24, 2012 by Hank Bromley

The old news Until about a year ago, if you wanted to upload a set of individual page images and have them be recognized as a “book” so we’d create the usual derivative formats from them, you had to mimic … Continue reading →

Posted in Books Archive, Technical | 1 Comment

Improved theora/ogg video derivatives!

Posted on May 1, 2012 by traceypooh

We’ve made our ogg video derivatives slightly better via: minor bump up to “thusnelda” release “upgrade” from 1-pass video encoding to 2-pass video encoding direct ffmpeg creation of the video (you’ll need to re/compile ffmpeg minimally with “–enable-libtheora –enable-libvorbis” configure … Continue reading →

Posted in Technical, Video Archive | Tagged , , , , | 3 Comments

Downloading in bulk using wget

Posted on April 26, 2012 by internetarchive

If you’ve ever wanted to download files from many different archive.org items in an automated way, here is one method to do it. ____________________________________________________________ Here’s an overview of what we’ll do: 1. Confirm or install a terminal emulator and wget … Continue reading →

Posted in Technical | 34 Comments

IA forums now do “inline reply”

Posted on February 11, 2012 by traceypooh

One thing that’s driven me a bit kooky is every time I “reply to this post” in the Internet Archive forums, it drives the browser to a new form page with no context/content of the post I was just looking … Continue reading →

Posted in Technical | Tagged | 1 Comment

new off-site video/audio embed codes

Posted on January 14, 2012 by traceypooh

We are about to rollout a “new new” video/audio player ?? You can see it in action now with our upcoming embed codes to go with this new player. It will allow for additional much wanted features like: – off-site … Continue reading →

Posted in Technical, Video Archive | Tagged , , , | 11 Comments

Open Hardware: Inexpensive Enclosures From Junction Boxes.

Posted on August 5, 2011 by daniel

I had a need for a cheap, standard enclosure for a humidity and temperature monitoring project. While there are many, many options for enclosures out there, few are cheap AND locally available. It occurred to me that electrical junction boxes … Continue reading →

Posted in Technical | 14 Comments

improved h.264 derivatives!

Posted on June 4, 2011 by traceypooh

We have thoroughly tested a newer and simpler way to create h.264 derivatives! Changes you’ll notice: More pixels! ?previously 320 x 240 ? ?goes to 640 x 480 pixels Slightly higher video bitrate — from about 512kb/s ? to ? … Continue reading →

Posted in Technical, Video Archive | Tagged , , , , , | 25 Comments

How Archive.org items are structured

Posted on March 31, 2011 by Alexis Rossi

What is an item? An item is a logical “thing” that we present on one web page on archive.org. An item may be one video file along with scans of the DVD cover, one book, one audio file, or a … Continue reading →

Posted in Technical | 7 Comments

How Montana State Library Uploaded Batches of Digital Objects to the Internet Archive

Posted on December 30, 2010 by internetarchive

by Chris Stockwell for Montana State Library, 12/29/2010 Introduction The Montana State Library (MSL) last year moved a copy of its collection of 3000 born digital state publications to the Internet Archive (IA). Since MSL will be continuing to upload … Continue reading →

Posted in Books Archive, Technical | 10 Comments