Course note: Coursera Machine learning by Andrew Ng, 2014,
week 10: Application example: photo OCR (https://class.coursera.org/ml-006/lecture)
When we are working on a machine learning task that is pipelined, how do we decide which componets are the most crucial ones for the improment of overall performance? Do a ceiling analysis!!
Let's say the pipeline is A->B->C->D,
the current overall performance is, say, 70%.
Then we manually label the part A, making A perfect, and see how much we can improve,
then make B perfect, see how much we can improve, and so on, ...
If making X perfect doesn't improve the overall performance significantly, then it's not worth devoting resources into.