This page describes the CE Linux Forum Open Test Lab, which is currently under construction.

Table Of Contents:

Contents

  1. Lab Resources - web site, wiki and mailing list
  2. Introduction and Purpose
    1. Usage Models
    2. Benefits of Testing
    3. Benefits of Remote Access to Boards
    4. Network Effects
  3. Architecture
    1. Lab architecture
    2. Architecture Diagram
    3. Lab Hardware
  4. Lab Software
    1. Lab Tests
  5. Lab Facilities
    1. San Jose lab (main lab)
    2. Tokyo lab (satellite lab)
  6. Lab Roles
    1. CELF Lab Director
    2. Lab Manager
    3. Board Support Engineer
    4. Test Developer
    5. Lab Software Developer
    6. Lab User
    7. Unassigned responsibilities
  7. Documents
    1. Lab Architecture Document
    2. Lab Manager's Guide
    3. Board Supplier's Guide
    4. Test Developer's Guide
    5. Lab User's Guide
  8. Lab Startup
    1. Phase 1 - basic lab operation
    2. Phase 2 - broader board solicitation and test suite buildup
    3. Phase 3 - more fancy stuff

Lab Resources - web site, wiki and mailing list

[ Material following this should be moved to test lab wiki from this main CELF wiki ]

Introduction and Purpose

This document describes the Open Test Lab of the CE Linux Forum.

The purpose of the lab is, at the highest level, the same as the purpose of the forum, which is to help enhance Linux and Linux technologies for use in Consumer Electronics products.

The lab exists to provide testing and quality assurance benefits to the entire Linux community, including CE Linux forum members.

The core of the lab is located in San Jose, but a satellite lab (an extension to the main lab) is planned for Tokyo, and additional lab nodes may be added to the lab as part of a completely distributed lab model.

The lab is available for use by CE Linux Forum members, and will be made available to open source community members as well.

Usage Models

The lab provides resources for a number of different testing activities:

Benefits of Testing

[It's good to test software. It makes it better. This part needs work, but it is pretty obvious.]

Benefits of Remote Access to Boards

The preferred mode of developing and testing software on a development board is to have the board in-hand (locally). However, sometimes it is not feasible for every interested developer to obtain a board. Sometimes development boards are expensive, or are made in limited quantities. This is especially true of prototype boards, which may have a limited development lifetime. Making a board available via the CELF test lab may help a developer who could not afford a development board, or who may not have priority to receive one.

In some situations, it is too much trouble to obtain a board when only a simple test needs to be performed or a small piece of data needs to be collected. For example, an indepedent developer might ordinarily have to perform a lot of tedious and time-consuming tasks in order to obtain a board. This might consist of waiting for necessary permissions, exchanging paperwork for a board loan, shipping, and having to do board-specific setup. For a quick test this overhead might not be worth the trouble. By lowering the barriers to access a development board, it is hoped that it will lead to increased use and testing on that board, by a wide variety of developers and for a wide variety of development tasks.

Not all development tasks can be performed under this model. For some tasks, such as evaluating audio or video performance or quality, it really is required to have a board locally. However, some tasks CAN be performed quite effectively remotely.

Network Effects

There are network effects (benefits which accrue from increased interconnections between parts of a system) which derive from the increased usability and availability of embedded development boards. A board supplier benefits from increased use (including development and testing) of a board that is available to a wider audience. For an embedded developer, it is useful to have access to boards that they would not otherwise be able to use. Also, when a variety of boards can be tested in a uniform manner, it helps to find bugs and problems in code. For many open source developers, even the ability to compile the same code on another architecture may help find bugs and problems in new code.

One of the most important reasons for the CELF Open Test Lab is to create an environment where these needs are met, and where the embedded Linux developers can reinforce their efforts to make progress on technology improvements for Linux.

Due to a number of factors unique to the embedded space, the community of developers that work on Linux for embedded products is fragmented. Through use of the test lab, it is hoped that this fragmentation can be reduced, and greater progress at improving Linux for this space can be made.

Architecture

Lab architecture

The documentation for the lab is divided into several different documents, as described below:

Architecture Diagram

Here is an architecture diagram of the CELF test lab:

http://tree.celinuxforum.org/CelfPubWiki/TestLabArchitecture?action=AttachFile&do=get&target=CELF-test-lab-diagram2.png

Lab Hardware

The hardware for the lab is described here. Some of this equipment has been purchased, and some target boards have been delivered already.

Her are some of the individual hardware items that will be used in the lab:

Additional information about the hardware for the lab is kept in the internal wiki for lab administrators.

Lab Software

Software for the lab is being developed now, and will be available from TestLabPrograms

Here are some of the individual software items needed for the lab.

Lab Tests

The test lab is building a repository of actual tests.

The list of currently available tests is at: LabTests

Lab Facilities

San Jose lab (main lab)

The main test lab is located at: {{{CE Linux Forum Test Lab Suite 171 1900 Wyatt Drive San Jose, CA, 95054 }}}

Tokyo lab (satellite lab)

We are working on creating a satellite lab in Tokyo, in order to provide ease-of-access to our membership in Japan.

We are currently pursuing negotiations with an outside party to utilize space in their facility. We will publish information about this facility as soon as it becomes available.

Lab Roles

CELF Lab Director

This person directs the overall operation of the lab, including coordinating contracts, obtaining lab facilities, managing lab personnel (where appropriate).

People:

Responsibilities:

Lab Manager

A lab manager is responsible for day-to-day operation of the lab. Most lab functions are automated, or are driven by user interaction. Therefore, the lab manager responsibility is primarily oriented to maintaining the operation of the lab, and fixing any problems that arise.

People:

Responsibilities:

Board Support Engineer

People:

Responsibilities:

Test Developer

People:

Responsibilities:

Lab Software Developer

People:

Responsibilities:

Lab User

People:

Responsibilities:

Unassigned responsibilities

Documents

Lab Architecture Document

(see above for now)

Lab Manager's Guide

Board Supplier's Guide

See TestLabBoardSuppliersGuide

Test Developer's Guide

Lab User's Guide

Lab Startup

Phase 1 - basic lab operation

Next steps:

Initial site plan:

For each target board in the lab, the following should be supported from the host machine for that board:

A board in this system would need to support:

Board provider must supply the following software: [need detail here]

For regressions, a download site for the kernel (or kernel patches on top of a base) must be provided.

Phase 2 - broader board solicitation and test suite buildup

Phase 3 - more fancy stuff

OpenTestLab (last edited 2008-05-07 18:22:20 by localhost)