Points, assumptions and more graphs!

Ok so it has been a busy week so here comes a list of what I have been up to with my dissertation. Don't worry there are pictures.

First thing I got working was MoG(Mixture of Gaussian) background subtraction using this page on OpenCV. This is to allow me to detect movement within the image that can help determine the likeliness of a person existing at that point. I have also implemented moving average background subtraction from this helpful site. I will be creating a test to try different values for moving average background subtraction to see which one is the best to use.

I have also implemented Cam shift which is starts off by doing a mean shift but then updates the size of the window and calculates the best rotation of a fitting ellipse to it. Then re applies the mean shift with the new scaled search window and previous window position. This is then repeated until the desired accuracy is met. (Paraphrased from here).


I am dealing with the problem of people being obscured by using a Kalman filter to determine the direction and velocity of people. From this I can determine if they are going to end up off screen or if they are still with in the scene but obscured. The issue may arise if there is an exit on screen that the person can disappear through. This issue can be dealt with by assuming that if the person isn't detected with in a number of frames then they are likely no longer in the scene. I won't be modelling the scene as that is a whole other project.


I have also made some more wonderful graphs to show the number of correct detections that each setting for the HoG detector provides. These can be seen below

Patterns include: Multiples of 4 having 0 detections and multiples of 6 having highest number of detections.

Steady drop of accuracy after scale setting of 1.01

This peaks at 3 but may go back up for higher numbers

My code is getting a little messier than I would like it to be. For this reason I will be going over my code and modularising it and doing some more testing for the new code I have done.


I am working on a points based system that will give points to the more accurate detections that are all consistent. This will use mean shift, cam shift and hog detectors to build up a more accurate person detector. I will create an array of ROIs(regions of interest) for each person depending on the score they get to build up what to look for.


I have encountered some problems along the way. Some of these include contours from background subtraction overlapping with other HoG boundaries meaning false positive when looking for movement in a HoG detection. Another issue is a 'non convex optimisation problem' with trying to search through the most accurate window stride for HoG detector. This is trying to avoid searching through all the possible combinations by going in the direction of the best detector settings in chunks.


An interesting point that was raised in a dissertation meeting was, should you count people that are occluded. If a person was counting number of people in a scene and a person was occluded that person would not be counted. There may be times when you want to know if a person is still in a scene even when obscured. An example would be when a place is limited to a certain number of people but they will be blocked by certain elements. In this case keeping track of number of people is important.

In the case of my dissertation I will be keeping track of all people that are judged to still be with in the scene.

I predict a riot

So finally decided on what my dissertation is going to focus on and, drum roll please, it is crowd behaviour. Specifically trying to count out when a crowd is likely to form, how many people are in a crowd, why they are they are in a crowd and dispersal patterns of the crowd. As well as this I will have to take into account the ethics of analysing people in crowds.

The main goals of this is to be able to determine at minimum number/groups of people in crowds. From this I can try to ascertain the situation and evaluate possible outcomes. It would be nice to be able to accurately count the people in a crowd however there are issues with trying to count people close together. If they are too close then an issue arises with counting many people as one person, as well as this the faces may not all be visible or clear enough to use facial recognition or just obstructed. There has been research into counting people in crowds or mapping high density crowds but I have not come across anything that is accurate and versatile between scenarios. The research linked for 'counting people in crowds' has a high success rate of more than 96% but the data set is only three videos. With regards to the link provided in the 'mapping high density crowds' they track motion against a certain threshold and if not over the threshold then it is considered static. This raises the issue of people who are not moving fast enough which can happen in over crowded areas would be considered background. On the other hand things, such as large animals, that move at the threshold could be used in the generation of the map thus making it not accurate. Being able to determine when a crowd is about to form would help determine things such as when a riot or a big event is about to happen( discussed in more detail later).

Below is an image of where people counting could be useful to make sure there is no over crowding. As well as this it could be used to determine best flow of traffic or if something abnormal is happening. *

Within crowds there are specific behaviours that can looked out for to help identify individual people or to try and work out what is going on in the scene. When walking down the street and another person is heading towards you the closer they get the more you move to the side to pass them as explained here. When people are coming from different angles but heading in the same direction they tend to merge in to a single flow of traffic. There is however, no specific detailed definition of a crowd but is defined as 'a large number of people gathered together in a disorganized or unruly way'. A few definitions do exist and share common specifics such as 'conceptualising a crowd as a sizeable number of people gathered at a specific location for a measurable time period, with common goals and displaying common behaviours' which is a extract from this online pdf. This document also goes into more detail about what is expected from a crowd. There is however issues with things such as determining when a group of people are considered a crowd, such as number of people and time spent together/at a location. For these reasons a set definition of a crowd must be made to allow a system to appropriately determine if there is a crowd or if a crowd is likely to form. As well as defining a crowd, definitions of crowds in different situations from data sets will allow the system to more clearly work out what is going on.

The accuracy of this is likely to be incrementally lower as the crowds get bigger as it will be harder to count people and monitoring the flow will become process heavy. As such focusing on each part such as the counting will allow more targeted results with hopefully higher accuracy. This could be useful though for predicting violence in crowds in which some work has been done here or even counting people in areas to avoid overcrowding and injuries.

Below image shows how the faces are not always showing on cameras, this makes it more difficult to count people using facial recognition. *

Now comes the fun part, ethics and what is ok to use, after all we will be looking at humans and their behaviour.The first thing we need to make sure is that the people who are on the video are ok to have themselves used for research purposes and their privacy is protected. Some questions need to be asked such as ones raised in this papers abstract. Questions such as 'Under what conditions should video be presented and to which audiences'. Videos that are used should have the consent of the people in it and should only be used for the purposes they are made for. As well as this there should be no attempt to try and identify persons within the videos unless that is the reason for the videos and you have express permission from all persons involved.


This is just a little overview of the parts of what needs to be discussed and will be discussed in more detail over the coming weeks.