Project Goals

  1. Digitizing current written test from end to end
  2. Digitizing student activities recording
  3. Reducing paper in printing the test and the student activity log sheet

Option Analysis and Feasibility Study

There are 4 options in project implementation.

First option is purchase existing product. Software procurement will cost a lot. Although some assessment systems are free to use, cost still exist because compulsory component may not be included. For instance, we need to set up our own database. Also, the flexibility is low because existing software may not fully satisfy user needs and modification may impossible.For example, Google forms module is not editable.Difficulty and time required in this option are low since main system is available already.

Second choice is building the system on our own with existing frameworks. This demands lowest level of cost because many frameworks are free to use. Although coding is easier, other difficulties still exist. For instance, choose feasible framework and understand how to use it is not simple. This development approach also consumes more time than full outsourcing. Furthermore, flexibility is not highest because some functions may be restricted by framework used.

Develop whole system on our own will be third option. Cost is higher when comparing to framework aided approach. It requires more manpower. Difficulty is highest because all technical issue will be solved by our own. Lots of time required as well. However, this offers highest flexibility since whole system is designed base on user’s requirement.

Last option is outsourcing thus new system will be developed by vendor. Like purchasing existing software, this may cost user a lot. Such option has high flexibility. The reason is vendor will also design and develop assessment system according user’s expectation. Also, this option has lowest difficulty since related risks and technical issues are transferred to vendor. However, project duration may not be shortened. Procurement steps such as writing Request for Proposal are required. After choosing vendor, we need to give reasonable time to vendor for project development according to budget. Also, we should reserve time buffer in case vendor completed the task late.

Analysis is summarized in below table.

Options Vendor existing
solution (1)
Develop own system with existing
library / framework (2)
Develop whole system
on our own (3)
New system developed
by vendor (4)
Cost High Low Mid High
Flexibility Low Mid High High
Difficulty Low Mid High Low
Time Low Mid High High

To obtain the feasible option, we also performed quantitative analysis with this scoring matrix. Positive factors give higher score.

High Middle Low
Cost 1 2 3
Flexibility 3 2 1
Difficulty 1 2 3
Time 1 2 3

Quantitative analysis result is shown in following table:

Options Vendor existing
solution (1)
Develop own system with existing
library / framework (2)
Develop whole system
on our own (3)
New system developed
by vendor (4)
Cost 1 3 2 1
Flexibility 1 2 3 3
Difficulty 3 2 1 3
Time 3 2 1 1
Total 8 9 7 8

After considering above criteria, option 2 was chosen because it is cheap and has the flexibility to implement user specific requirement. The time required can also fit the project schedule.


Project Approach

We will adopt a waterfall approach in the project which user requirements will be clarified before starting development. Project scope must be defined before development starts and the change in scope will be carefully managed by the project manager.


Project Scope and Requirement Prioritization

Here are expectations from users in this project.

  1. Assist test paper preparation based on question bank
    There will be a question bank in system and leaders will design question paper from questions in the bank.
  2. Member can do the test with their own devices.
    The assessment system should be able to work on different platform.
  3. Provide test paper printing function
    Test paper set in system can be printed out for any contingency. For instance, if the network is down, test can conduct on time with hard copies.
  4. Arbitrary order of questions
    Question order in same test paper among students should be different.
  5. Mark answers automatically
    System should able to perform test grading automatically. For typing questions, grading module should able to detect and omit minor typo mistakes.
  6. Leaders can edit marking of the system
    After system marking, users should be able to update the test result.
  7. Record members’ other accomplishment records
    Achievements of members, including test result and qualifications, should be centralized in this system.
  8. User-friendly interface
    No special training required before using this system. For members,
  9. Support different types of questions
    Multiple choices and fill in the blanks as well as short questions should be included in system. Besides, new type of questions designed by users should be accepted.
  10. Centralize teaching resources
    User states that no centralized platform for teaching resource sharing.
Requirement Rationale
Core Functions
1 Assist test paper preparation
based on question bank
This is the most important function
of the assessment system
4 Arbitrary orderof questions Simple function implementation
and able to increase satisfaction
5 Mark answers automatically This feature is valuable because
it saves lot of leader manpower
on test paper marking
6 Leaders can edit marking
of the system
Edit marking is another core function
due to automatic marking is included
8 User-friendly interface Simple interface must be provided.
Different types of users should able to
use this system without special training
Lower Priority Core Functions
2 Member can do the test
with their own devices
Cross platform is time consuming
in both coding and testing.
3 Provide test paper
printing function
Uncertainties exist when constructing printing
function with various printers and encoding.
9 Support different types
of questions
Flexibility on question type is important
but may require large amount of resource.
Special design required to deal with unseen
question types
Non-core Functions
7 Record members’ other
accomplishment records
Centralizing all members’ achievements is
not main purpose of assessment system.
So, this will be constructed when extra
resource available and core functions are
completed.
Excluded functions
10 Centralize teaching resources This function is excluded because not
closely related to assessment system
and may be time-consuming.Different from
centralization of members’ achievements,
this feature is more complicated due to
teaching material are in different forms

Besides above expectations from user, we propose following extra functions:

  1. Test result report and analysis
    We purpose an assessment result analysis. Various kind of analysis is available after test result digitalization. Analysis on question with lowest correct rate and member performance are examples. This feature helps users in question paper design.
  2. Question frequency detection
    User concerned about frequent use of same questions in different course assessments. Thus, question frequency detection function is suggested. There will be a threshold of question appearance maintained by user. This module will count question occurrence after selection from bank. In case appearance count exceeds threshold, system will show reminder.
  3. Automatic question refreshment
    Sometimes question may be unclear or contains typo mistake. We propose a function that question displayed in member’s devices will be updated at once after leader’s modification. Therefore, leaders can update questions during test.