Processing data at the UK Data Archive

Once a data collection is received by the UK Data Archive (via the ESDS or the History Data Service), the following steps are undertaken.

Deposit review

Following acknowledgment of receipt of the data collection a deposit review is conducted. Checks are carried out to ensure that:

• the deposit is complete, i.e. all required materials have been enclosed
• the data collection has no ethical, legal or rights issues which might prevent sharing
• the condition and format of the dataset are suitable for secondary use and long-term preservation
• documentation is sufficient to enable secondary use of data

All data depositors are expected to have prepared their data following the UK Data Archive's guidance on how to manage and share data.

In-house processing

After the inital review, the data collection is further assessed by the UK Data Archive's Data Services section before full processing begins. The data collection is examined in detail and decisions are made regarding file preservation and dissemination formats, composition of documentation and levels of validation. For more detailed information on in-house data processing see Archiving data at the UK Data Archive.

Releasing the data

Once processing is complete, all physical materials associated with the study are placed in secure storage and data and documentation become available for registered users to download, order or browse online. A release notification letter is sent to the depositor.

Service Level Definition (SLD)

A Service Level Definition (SLD) for processing data collections is in place for data coming in through ESDS. This acts as a quality assurance measure for both the depositor and the Data Services team. It ensures that processing is completed and data are made available within a certain time limit, but also that sufficient time is available for thorough processing of the data. Current SLD time limits are as follows:

• quantitative data processing: 30 working days - where the data collection is destined for Nesstar an additional five working days are allowed in order to accommodate the extra preparatory work involved
• qualitative data processing: 35 working days - qualitative data processing often takes more time than quantitative, depending on the size and complexity of the data collection

Should a problem subsequently be discovered during processing that requires further contact with the depositor, work may be suspended at any time during the 30/35 day SLD period. Timed processing will resume once the problem has been resolved.

In practice, the majority of data colections are processed very quickly. The efficiency of ESDS acquisitions procedures ensures that most deposits are in excellent condition on receipt and the initial review uncovers most potential problems. However, some issues may not become apparent until detailed work begins, necessitating suspension of processing while the depositor is contacted for clarification.

Depositors may in some cases need to know in advance when their data collection will be ready for secondary use. It should be noted that whilst every effort is made to ensure that processing is completed quickly, ESDS cannot guarantee a date other than that dictated by the 30/35 day SLD time limit, for the reasons outlined above. Depositors can contact ESDS at any time for clarification on progress. If depositors have already received enquiries from potential users, they can direct the user in the first instance to help@esds.ac.uk.