ML Cost - Yousef's Notes
ML Cost

ML Cost

#Difficulty of the Problem

  • Do we have ready made ML solutions (libraries, systems)?
  • Depending on the problem, which type of algorithms do we need to design/implement (classification, regression, clustering, or reinforcement learning) and how difficult is it to implement/deliver/maintain?

#Cost of the Data

  • Can data be generated automatically?
  • What is the cost of manual annotation of the data?
  • How many examples are needed? Cannot be known in advance, but can be estimated from known published results or the organization’s knowledge.

#Need for Accuracy

  • Cost grows super-linearly with the accuracy requirement.
  • How costly is each wrong prediction?
  • What is the lowest tolerable accuracy level below which the model becomes impractical?