We are interested in studying the application of computer vision and visual clusters on creating both foreground and background objects. This is motivated by the observation that deep neural networks learn context as part of the objects. Therefore, by having a dictionary of parts for both foreground and background objects we can capture objects in occluded settings and separate the foreground from the background. Our work makes further contribution by having one network for both foreground and background objects rather than having multiple networks for multiple set of objects. We have demonstrated that a combination of the right network architecture and clustering algorithm can create visual concepts that can be used for both foreground and background objects.