INFS2044 2 Case Study

INFS2044 Assignment 2 Case Study

In this assignment, you will be developing a system for finding images based on the objectspresent in the images. The system will ingest images, detect objects in the images, and

retrieve images based on labels associated with objects and by similarity with an exampleimage.Use CasesThe system supports the following use cases:

• UC1 Ingest Image: User provides an image, and System stores the image, identifiesobjects in the image, and records the object types detected in the image in an index.

• UC2 Retrieve Objects by Description: User specifies a list of object types, and thesystem returns the images in its index that match those listed. The system shallsupport two matching modes:

o ALL: an image matches if and only if an object of each specified type ispresent in the imageo SOME: an image matches if an object of at least one specified type is presentin the image

• UC3 Retrieve Similar Images: User provides an image, and the system retrieves thetop K most similar images in order of descending similarity. The provided image mayor may not already be in the system. The similarity between two images isdetermined based on the cosine similarity measure between the object typespresent in each image. The integer K (K>1) specifies the maximum number of imageso retrieve.

• UC4 List Images: System shows each image and the object types associated with

The following are example commands that the command line frontend of the system shallimplement:UC1:

$ python image_search.py add example_images/image1.jpg

$ python image_search.py listexample_images/image1.jpg: chair,dining table,potted plantexample_images/image2.jpg: car,person,truckexample_images/image3.jpg: chair,personexample_images/image4.jpg: carexample_images/image5.jpg: car,person,traffic lightexample_images/image6.jpg: chair,couch6 images found.Other requirementsInput File FormatThe system shall be able to read and process images in JPEG format.For UC2, you can assume that all labels are entered in lowercase, above.Unless indicated otherwise, the output of the system does notneed to be sorted.For UC3, the output shall be sorted in descending order of similarity. That is, the mostsimilar matching image and its similarity 代写INFS2044 2 Case Study  shall be listed first, followed by the next similarimage, etc.For UC4, the output shall be sorted in ascending alphabetical order.Internal StorageYou are free to choose either a file-based storage mechanism or an SQLite-based databasefor the implementation of the Index Access component.The index shall store the file path to the image, not the image data itself.Object detectionThe supplied code for object detection can detect ~90 object types.uture variations

• Other object detection models (including external cloud-based systems) could beimplemented.

• Additional object types could be introduced.
• Additional query types could be introduced.
• Other similarity metrics could be implemented.
• Other indexing technologies could be leveraged.
• Other output formats (for the same information) could be introduced.These variations are not in scope for your implementation in this assignment, but yourdesign must be able to accommodate these extensions largely without modifying thecodethat you have produced. DecompositionYou must use the following component decomposition as the basis for your implementationdesign:The responsibilities of the elements are as follows:

Elements

Responsibilities

Console AppFront-end, interact with the userImage Search ManageOrchestrates the use case processesObject Detection EngineDetect objects in an imagMatching EngineFinds matching images given the object typesIndex AccessStores and accesses the indexed imagesmage AccessRead images from the file systemYou may introduce additional components in the architecture, provided that you justify whythese additional components are required.Scope & ConstraintsYour implementation must respect the boundaries defined by the decomposition andinclude classes for each of the elements in this decomposition.The implementation must:

• run using Python 3.10 or higher, and
• use only the Python 3.10 standard libraries and the packages listed in therequirements.txt files supplied with this case study, and

• not rely on any platform-specific features, and
• extend the supplied code, and
• correctly implement the functions described in this document, and
• it must function correctly with any given input files (you can assume that the entirecontent of the files fits into main memory), and

• it must include a comprehensive unit test suite using pytest, and
• adhere to the given decomposition and design principles taught in this course.Focus your attention on the quality of the code.It is not sufficient to merely create a functionally correct program to pass this assignment.The emphasis is on creating a well-structured, modular, object-oriented design that satisfiesthe design principles and coding practices discussed in this course.Implementation NotesYou can use the code supplied in module object_detector.py to detect objects inimages and to encode the tags associated with an image as a Boolean vector (which you willneed to compute the cosine similarity). Do not modify this file.You can use the function matplotlib.image.imread to load the image data from a file, andsklearn.metrics.pairwise.cosine_similarity to compute the cosine similarity between twovectors representing lists of tags