Directly Operable Image Representation of Multiscale Primal Sketch

In this paper, we propose a versatile semantics-driven image representation that can support many common operations in visual computing and communications, in addition to being itself an efficient image coding scheme. The proposed image representation is based on a semantically meaningful construct called multiscale primal sketch (MPS). The MPS consists of edges that are extracted and organized successively from fine to coarse scales. The edges are further classified into two types: pulse edge and step edge. MPS is an intermediate-level image representation, which is between pixel-based (low-level) and object-based (high-level) descriptions. The MPS image representation reaches a good compromise between its construction cost and descriptive power. It has a compact form, and hence is amenable to image compression. Furthermore, because the new representation consists of semantically meaningful primitives—edges of different scales and types, and background—many common image operations, such as classification, restoration, detection, and content-based retrieval, can be performed directly in the MPS framework, without first converting the coded image back to the spatial domain.