Strategies to Detect Neutrinos

What are Neutrinos and how do we detect them
A neutrino (ν) is a subatomic particle from the lepton family with a lepton number of +1, a charge of 0 and a spin of ½. There are three flavours of neutrino the Muon Neutrino, Tau Neutrino and the Electron Neutrino1. Neutrinos rarely interact with matter because they are so small and have no charge and are also not affected by the strong nuclear force. So the only way a neutrino can interact with matter is through the weak nuclear force. Neutrinos are about 100,000 times smaller than electrons but there are so many neutrinos being emitted into the universe that even with their incredibly small mass they outweigh the amount of matter in the universe9.

Figure 1
Neutrinos were hypothesised in 1930 by Wolfgang Pauli8, he theorised that another particle must be emitted in beta decay other than the electron as not all the energy from the decay carried by the electron so Pauli suggested that another particle was emitted and was carrying the rest of the energy given off. It was expected that the electron would carry all the energy but this is not what was found. The law of conservation of energy states that Energy can’t be created or destroyed, but it can be changed into a different form, also that in a closed system it cannot be lost.
The red line represents the energy the electron should have if none was shared from the beta decay of carbon 14 and the blue line represents the actual energy of the electrons.

The first people to detect the neutrino were Reines and Cowan. They did this by using the prediction the nuclear reactors were meant to produce high amounts of neutrino fluxes. When one of the antineutrinos collides with a proton a neutron and a positron are given off6.
These positrons then collide with electrons and annihilate via pair-annihilation. When this happens two gamma rays are produced as radiation in opposite direction.
Figure 8

Reines and Cowan soon realized that detecting the gamma bursts wasn’t enough evidence to categorically say they had found neutrinos. So they aimed to detect the neutron given off as well. Reines and Cowan set up a new experiment where they constructed a tank of water and lined it with a scintillating material to detect the gamma radiation. A scintillating material is a material that fluoresces when hit by a photon or a charged particle. This is then picked up and amplified by photomultiplier tubes. They also put cadmium into the tank; cadmium absorbed the neutrons given off in the reaction between the antineutrino and the proton and becomes an exited form of cadmium witch give off gamma radiation1.1.
Figure 9
The gamma rays form the exited cadmium were detected 5X10-6 seconds after the positron electron annihilation. This gave enough evidence to prove that neutrinos did exist. Reines and Cowan repeated the experiment in a different location with better cosmic ray shielding. Cosmic rays comprise of very high energy particles such as high energy photons, these particles can interfere with very sensitive electronics used in the experiments and can create false readings. Form this they got more reliable results1.1.
In a reaction the baryon number, lepton number and the strangeness must stay the same. So in beta decay where an electron is given off an anti lepton must be released to make the lepton number 0 again.
 
 
 
 
 
 
 
 
I am going to be looking at how the Super-Kamiokandeis able to detect neutrinos. The Super-Kamiokandeis a large experiment where 50,000 tonnes2 of ultra pure water is held in a stainless steel spherical tank covered in 11,146 photomultiplier tubes all of this is located in a old mine 1,000 meters underground to stop cosmic ray interference. To be detected, a neutrino would interact with a H2O molecule and would cause an Electron to be discharged and this would be travelling faster than the speed of light in water causing Cherenkov radiation to be emitted. Cherenkov radiation is emitted when a particle travels faster than the maximum velocity of a photon in that medium. This radiation produces a ring of light which is detected by the photomultiplier tubes witch amplify the signal, using this we can calculate where the neutrino interacted and what flavour of neutrino it was3.

Photomultiplier tubes are needed as they are able to amplify the signal by around 100 million times. When a photon from the Cherenkov radiation hits the photocathode then a photoelectron is released vie the photoelectric effect , this is then attracted to the first dynode with a pd of approximately 100V this electron gains kinetic energy and then hits the dynode liberating more electrons (typically 3-4) then these are attracted to the next electrode with a pd of 100V and a charge of 200eV and the same happens again until there is a strong enough signal and the electrons hit the anode and then the detected signal is sent off to the computer4.
 
For each electron liberated on the dynodes the energy is

The 100eV the electron carries is enough to liberate around 4 new electrons on the next dynode.

With some electrons not hitting the dynodes and some not liberating exactly 4 new electrons then the figure that the signal is amplified by 100 million times and that (3-4) electrons are liberated by on electron make are correct.
From research it seems that the dynodes have a work function (ψ) of around 5eV this means that about 80eV is lost when the electron hits the surface of the material.

Figure 5
This Is the Super-Kamiokande form the inside. Each dot is a photomultiplier tube, and there are two people checking them on the surface of the water in a dingy.
The first recorded instance of an observation of a neutrino was in 1970 on the 13 of November. The event was observed when a neutrino collided with a proton and created a mu-meson (muon) and a pi+-meson (pion). A pi+-meson is a particle which consists of a quark and an anti-quark. A pi+-meson consists of an up quark and an anti down quark. A muon is a member of the lepton family in the standard model. This all occurred in a hydrogen bubble chamber. A bubble chamber is a vessel that holds super heated liquid (in this case hydrogen); it is used to detect charged particles that enter it. It is able to crate observations of these particles as when a charged particle passes through the chamber it causes an ionisation path which causes the surrounding liquid to vaporise and form bubbles which size are proportional to the specific particles energy loss. This is all captured by cameras which can produce a picture of the event5.

Figure 6
This is the original picture of the collision

This is an annotated picture showing the paths of the colliding particles. Muon (μ–), proton (p), neutrino (νμ) and the pion (π+). When the neutrino and the proton collide the proton moves to the left. The neutrino is turned into a muon which keeps going forwards and the pion is created from the collision5.
The annotation to the right shows what is happening at the sub atomic level with quarks.
In 2011 the OPERA experiment conducted which came across the odd results that neutrinos were travelling faster than the speed of light. The results were declared as anomalous as anything going faster than the speed of light in a vacuum is considered to go against special relativity. The scientists conducting the experiment set investigations into why they got the results they did. From these investigations it was found the there were two faults in how the experiment was set up. One was that a fibre optic cable was improperly connected and that a clock oscillator was set to fast. Taking both of these errors into account meant the reading were not actually faster than the speed of light.
In 2012 it was reported that the speeds of neutrinos are the same as the speed of light. This information was gathered by numerous different scientific groups including OPERA.
There are many different sources of neutrinos such gamma ray bursts, supernovas, neutron stars, nuclear fission and cosmic rays. Neutrinos are defiantly not rare with potentially about 100,000 billion passing through your body every second. All of these sources are some of the most energetic/violent processes in the universe. The main source of our neutrinos that are detected by places like ice cube and Super-Kamiokande is the sun through its nuclear fission which gives off many neutrinos.

Here you can so that a neutrino and a positron are emitted when two H1 atoms collide and coalesce to form a H2 atom.

Ice cube is another neutrino detector in the South Pole that uses the same idea as the super-Kamiokande in that it detects the gamma rays from when a neutrino collides with a water molecule. Ice cube is a hexagon that is around 2,450 meters deep and has 86 lines of sensors with 60 sensors on each line so a total of 5,160 sensors.
Conclusion
From my research into what neutrinos are and how we can detect them I have found out the fundamental nature of neutrinos and how we are able to detect something that rarely interacts with matter. I have learnt that neutrinos are harder to detect than I had imagined and that there are different methods such as detecting the Cherenkov radiation from the neutrinos colliding with water molecules or by seeing their ionising path in a bubble chamber. I have also found out some of the reasons behind why neutrinos are so hard to detect in the first place, for example that neutrinos are extremely small, have very low mass, are not charged and only really interact through the weak nuclear force. Over all, neutrinos are very elusive and one of the weirder particles that we have discovered and there is still a lot we do not know about them.
Bibliography

Date accessed: 23/11/2014

1 URL: http://hyperphysics.phy-astr.gsu.edu/hbase/particles/lepton.html

URL: http://hyperphysics.phy-astr.gsu.edu/hbase/particles/cowan.html

Hyper physics is a reliable website source because it is hosted by the physics and astronomy department at Georgia state university and has professors who teach the subjects input also it should be non biased as there is no gain for it providing false information. Hyper physics states that their second experiment at Savannah River Plantwas 12 meters underground and states the cross-section of the reaction to be 6X10-44 and the same figures are stated http://en.wikipedia.org/wiki/Cowan%E2%80%93Reines_neutrino_experiment

Date accessed: 23/11/2014

URL: http://physicsworld.com/cws/article/news/1998/jun/05/super-kamiokande-finds-neutrino-mass
Physics world is a website that publishes the new and old physics topics and has many different topics that it has published. It is a reliable source as it is backed by some very credible companies, such as Angstrom Sciences and Moxtek Inc. It also has scientist informing and righting as well which further proves the reliability of the website.

Date accessed: 21/11/2014

URL: http://www.princeton.edu/~achaney/tmve/wiki100k/docs/Super-Kamiokande.html
Princeton.edu states that the page is sourced from Wikipedia
URL: http://en.wikipedia.org/wiki/Super-Kamiokande. it is reliable as Princeton university would not be publishing wrong information on their site as that would be bad for them so that gives this information some credibility. Also physics world URL:http://physicsworld.com/cws/article/news/1998/jun/05/super-kamiokande-finds-neutrino-mass states that the super-Kamiokande is 1000m underground and holds 50,000 tonnes of water which is the same as on the Princeton page this back up the reliability of the data.

Date accessed: 23/11/2014

URL: http://en.wikipedia.org/wiki/Photomultiplier
The data that I found on Wikipedia on photomultiplier tubes was backed up from the equations I used to try and estimate the number of electrons hitting the anode, which gave similar figures to my calculations. Also the theory behind how photomultipliers work was the same as explained in http://micro.magnet.fsu.edu/primer/digitalimaging/concepts/photomultipliers.html
this website also stated gains around 100 million which is my calculated and Wikipedia’s stated value. All this shows that it is a reliable source.

Date accessed: 26/11/2014

URL: http://abyss.uoregon.edu/~js/ast123/lectures/lec22.html
This is an educational site from the University of Oregon who should not be biased as they have no reason to put incorrect information on their website as it would have a negative effect on them and they wouldn’t gain anything. It is reliable as it is written by scientists. The date stated on the page November the 13th 1970 is the same as stated on http://commons.wikimedia.org/wiki/File:FirstNeutrinoEventAnnotated.jpg.

Date accessed: 26/11/2014

URL: http://t2k-experiment.org/neutrinos/a-brief-history/
T2k is a website dedicated to neutrinos. The website is primarily about news in the field and the t2k experiment of neutrino oscillation. It is a reliable source as it is written by professionals.” the positron annihilates with an electron to create two gamma rays” this statement says the same thing as http://hyperphysics.phy-astr.gsu.edu/hbase/particles/lepton.html says on the topic.

Date accessed: 30/11/2014

URL: http://icecube.wisc.edu/science/depth
Ice cube is a website dedicated to the ice cube particle detector in the south-pole that is trying to detect neutrinos and more. Its primary funding source is the national science foundation, this is a US government organisation that funds and conducts many different projects. Their aim is to keep US science at the forefront of the world in discovery. The web site ice cube should be reliable as it has major government input and would not gain anything from false publication. On ice cube it states that the detector has 5,160 detectors this is the same at http://phys.org/news/2013-11-world-largest-particle-detector-icecube.html. Phys is a large physics news blog with articles written by universities and scientists so it is a reliable website as it is written by people who have extensive knowledge in what they write.

Advanced Physics”, Steve Adams, Jonathan Allday/oxford university press/November 2nd 2000, p416

Advanced physics is a book published by oxford university press. It is reliable because Oxford University is a highly regarded university that would get a negative publicity if what they published was incorrect. Oxford should not be biased as it doesn’t have any large
Companies or influential people pressuring them to publish false information.

“Neutrino”, Frank Close/oxford university press/ February 23rd 2012, p2

The book neutrino talks about what neutrinos are and how we detect them, their history, their discovery, their sources and many different topics related to them. The point of the book is to inform and educate people on neutrinos. Professor Frank Close the author is a professor at Oxford University this shows he knows what he is talking about and that the book is reliable as he is a regarded physicist. Oxford press is a reliable publisher as I have stated in reference 8.
Figures

Date accessed: 23/11/2014 URL:http://upload.wikimedia.org/wikipedia/commons/thumb/0/00/Standard_Model_of_Elementary_Particles.svg/2000px-Standard_Model_of_Elementary_Particles.svg.png
Date accessed: 26/11/2014

URL: http://t2k-experiment.org/neutrinos/a-brief-history/

Date accessed: 23/11/2014

URL: http://hyperphysics.phy-astr.gsu.edu/hbase/particles/lepton.html

Date accessed: 23/11/2014

URL: http://www.ocr.org.uk/Images/165501-unit-g495-advance-notice-june-2014.pdf

Date accessed: 21/11/2014 URL:https://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAUQjhw&url=http%3A%2F%2Fpprc.qmul.ac.uk%2F~still%2Fwordpress%2F%3Fpage_id%3D138&ei=iSBnVK2IG5XWaqubgfAJ&bvm=bv.79142246,d.d2s&psig=AFQjCNH_tc4ZUVMJfiVeSgUvb3ba_uDsqA
Date accessed: 26/11/2014 URL:https://www.windows2universe.org/sun/Solar_interior/Nuclear_Reactions/Neutrinos/neutrino_picture_big.gif
Date accessed: 26/11/2014

URL:http://abyss.uoregon.edu/~js/ast123/lectures/lec22.html

Date accessed: 23/11/2014

URL:http://hyperphysics.phy-astr.gsu.edu/hbase/particles/cowan.html

Date accessed: 23/11/2014

URL:http://hyperphysics.phy-astr.gsu.edu/hbase/particles/cowan.html
 

Machine Learning Application using Pose Estimation to Detect and Moderate Violence in Live Videos

Abstract

Recordings of public violence have never been as readily available as today. Livefeeds of shootings and attacks have become an ever increasing problem with gruesome images of violence being a click away from viewers of all ages. AI has begun to be employed to monitor video surveillance in prisons or psychiatric centres to detect “suspicious behaviour” but this technique has yet to be exploited for more general monitoring live broadcast and media sharing sites such as Twitch.tv and Facebook live. This proposed model could be useful as a “missing piece” in the field of censorship AI and used as the basis of a start-up company, as a web browser ad-on or sold directly to streaming services to be incorporated into their website. 

Overview

A significant amount of research has be done on the various methods to detect violence in videos, focusing on visual content,[1] audio content[2] or a combination of the two.[3] There has been major success with real-time monitoring of audio profanities and nudity but on most online platforms to date manual human monitoring and reporting is still relied upon to detect violent content.[4] This report will focus on violent human behaviour such as fighting as opposed to videos involving weapons, blood or fire which have been previously classified using simple image classification algorithms.[5]

Violent human behaviour can be classified in real-time using pose estimation, an emerging area of research where 3D stick-man poses of individuals can be extracted from 2D pictures and videos. Some of the numerous current applications include automatic creation of assets for digital media such as video games, analysing and coaching the techniques of athletes and with specific interest to this report, machine learning using image classification techniques.[6] Difficulties in the process include accounting for lighting, occlusion and variety of clothing. An advantage of using deep learning over “hand-crafted” techniques is the lack of the need for generalisation of frames and prior information means there is no need to heavy pre-processing of the data.

2.1.  Previous developments on pose classification 

Bogo and Kanazawa et al. previously reported a convolutional neural network, used to predict the position of individual joints which can be used as a basis to use a Skinned Multi-Person Linear Model optimisation in a classic bottom up – top down approach where the full 3D-geometry and body type is also conferred.[7] The computational demand of this approach is optimised by using constraints such as avoiding impossible join bends, thus minimising the number of possible solutions.

Get Help With Your Essay
If you need assistance with writing your essay, our professional essay writing service is here to help!
Essay Writing Service

Real-time pose estimation has been achieve with the work of Güler et al.[8] The technique of dense human pose estimation maps human pixels in a frame of a video to the 3D surface of the human body with positive results, improved by training an additional “inpainting” network that filled in missing values based on the surrounding data. Opposed to previous research of Bogo and Kanazawa et al., the output poses are dense correspondents between the 2D images and the 3D models, named DensePose.

  System description

The objective of this report was to develop a model for an end to end trainable, deep neural network to classify live videos to detect violence. The specific aims can be broken down as follows:

●       Assess the performance of employing a convolutional neural network to train on frame differences and a ConvLSTM to classify the frames where the output gives an overall probability violence score ranging between zero for extremely unlikely to be violent and 1 for certain violence.

●       Theorise the pros and cons of the system as well as performing an evaluation the algorithm against known benchmarks.

●       Detail how this model can be incorporated into a live-video moderation application for streaming sites and browsers.

3.1.  Pose classification

A convolutional neural network is used to extract frame level features using frame difference from a video in real-time. The output of the trained convolutional network, which will be the desired pose information will be subsequently fed into a seriesfeed-forward layer to output the probability and thus level of violence in the frames thus far. The model can be considered a classical blackbox of which a block-diagram can be viewed in Figure 1.

Although an advantage of convolutional neural networks is an absence of extensive pre-processing of the training data, the method in which the video frames are fed to the model can improve the accuracy of the algorithm. Classification accuracy was investigated by Krizhevsky et al. for the ImageNet dataset using each video frame separately and the difference between each frame.[9] The classification accuracy rose from 96.0

The recognition accuracy and error rate of the algorithm can be evaluated on a number of standard benchmark datasets, for example the Hollywood, YouTube-Actions and Violent-Flows dataset. This can be performed using cross validation against other well-studied classification techniques such as ViF/SVM,[12] OVif[13] and MoSIFT[14] that have been evaluated on the Violent-Flows dataset. An extremely robust dataset of pose-action classification known as Action Similarity Labelling (ASLAN) was presented by Kliper-Gross et al. in 2012 and has become a standard benchmark dataset for pose estimation in the years following.[15]  The dataset includes thousands of videos, collected incorporating over 400 complex pose-action with violent and non-violent classes. Models incorporating both convolution and ConvLSTM layers for pose classification exhibit accuracies upwards of approx. 94 % when evaluated against Violent-Flows and ASLAN.[4] From this similar results should be expected from the proposed model present in this report. As mentioned in Section 3, problems have arisen in previous work when including videos of sporting events into the training dataset. When evaluating this model, accuracy values should be taken when both including and excluding sports footage.

  Discussion: Application of the algorithm

The final output of the model described in section 3 gives a user a continuous probability score for the presence of violence in a livestream in real time. The model has been developed to incorporate modern methods such as ConvLSTM to improve classification accuracy and blackbox convolutional neural networks to allow for real-time detection. Assuming that the model performs well against established benchmark datasets, the next step is to research the algorithm’s viability as a commercial product and specify the niche market, functionality and obstacles that a start-up company using this technology may face.

5.1.  Market analysis

The use of deep neural networks for video violence detection applications is currently in its infancy. The most prominent use of the technology to date is seen in the “AI Guardman” developed by the company Earth Eyes released in late 2018. The software boasts the ability to target shoplifting using CCTV using a post estimation model based on OpenPose, a predecessor to DensePose discussed in Section 2. Although the source code is not available, knowing that the product is largely based on the OpenPose algorithm infers that the algorithm cannot compute in real-time. To combat this only a selected number of poses are defined to reflect “suspicious behaviour”, leading to more inaccurate results, and increasing the number of false positives. The software occupies a niche as it does not require sound, something that standard CCTV cameras do not process. The software can be installed on the CCTV directly and alerts are sent to a shop-workers phone, who can then handle the rest of the matter. The software is simple but currently is plagued with false alarms. As all commercial uses of violence detection are geared towards surveillance, a niche for violence detection for streaming services is identified, optimised for online applications.

5.2.  Implementation

After identifying this algorithm as a unique product, it is important to understand how to implement the model in a valuable product. At this point it is critical to note that violence detection software to date has occupied surveillance monitoring as crude analysis can be tolerated as it works as a warning system which then can be followed up by human investigation. Due to this it has not had the need to incorporate other elements such as audio classification.

On its own this algorithm will be able to moderate livestreams based on action recognition, but when paired with already well-researched audio profanity and nudity detection, provides the missing piece to a robust streaming video moderator. Much work has been previously been done on the detection of audio profanities in videos, most notably Bleep developed for iOS released in 2015, which has the ability to censor swear words from voice calls and videos. The same can be said for nudity detection with NudeNet, released for video censoring in March 2019. Amalgamation of these three technologies results in a much requested feature for steaming services such as Twitch.tv which censor and report streams instantaneously.

As mentioned above websites such as Twitch.tv could benefit from this technology and a first application of a start-up company would be to pitch the idea to streaming services for incorporation into their website and/or app. The feature could be toggled for adult users and made mandatory for kid-accounts. The output of the model gives a probability score for the violence, so thresholds could be put into place. In addition to this a browser add-on could be developed for unsupported streaming websites.

Find Out How UKEssays.com Can Help You!
Our academic experts are ready and waiting to assist with any writing project you may have. From simple essay plans, through to full dissertations, you can guarantee we have a service perfectly matched to your needs.
View our services

The initial tasks of a start-up company would then be to incorporate a model which could have audio profanity and nudity detection algorithms running in parallel while still having the ability to detect frames in real-time. The idea could then be pitched to established streaming websites, who could host the algorithm server-side. In addition to this if there was demand for the product, a browser ad-on could be developed, incorporating a user-friendly interface with customisable censoring options such as only nudity censoring or censoring of violence above a certain probability threshold.

5.3.  Conclusion

Overall a literature review of pose-estimation and violence detection was conducted to present the notable research in the fields but the lack of a commercial application aside from surveillance. A model was proposed to use post estimation to detect violence in real-time, comprised of a convolutional neural network and LSTM-divided layers based on current research. The system architecture was discussed including a complete block-diagram for the system. Pros and cons for the algorithm were theorised along with a proposed system evaluation. Finally discussion was made on the capability of the algorithm to act as a steaming moderator and act as the product of a start-up company.

 References

[1] P. Bilinski, F. Bremond, I. S. Antipolis, R. Lucioles, and S. Antipolis, “Human Violence Recognition and Detection in Surveillance Videos,” AVSS, August, 2016.

[2] T. Giannakopoulos, A. Pikrakis, and S. Theodoridis, “A multi-class audio classification method with respect to violent content in movies using Bayesian Networks,” in 2007 IEEE 9Th International Workshop on Multimedia Signal Processing, MMSP 2007 – Proceedings, 2007, pp. 90–93.

[3] E. Acar, F. Hopfgartner, and S. Albayrak, “Breaking down violence detection: Combining divide-et-impera and coarse-to-fine strategies,” Neurocomputing, vol. 208, pp. 225–237, Oct. 2016.

[4] S. Sudhakaran and O. Lanz, “Learning to detect violent videos using convolutional long short-term memory,” in 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2017, 2017.

[5] O. Arriaga, P. Plöger, and M. Valdenegro-Toro, “Image Captioning and Classification of Dangerous Situations,” 2017.

[6] M. Ariz, A. Villanueva, and R. Cabeza, “Robust and accurate 2D-tracking-based 3D positioning method: Application to head pose estimation,” Computer Vision and Image Understanding, vol. 180, Academic Press, 2016, pp. 13–22.

[7] F. Bogo, A. Kanazawa, C. Lassner, P. Gehler, J. Romero, and M. J. Black, “arXiv : 1607 . 08128v1 [ cs . CV ] 27 Jul 2016 Keep it SMPL : Automatic Estimation of 3D Human Pose and Shape from a Single Image,” eccv2016, 2016,  pp. 1–18.

[8] R. A. Güler, N. Neverova, and I. Kokkinos, “DensePose: Dense Human Pose Estimation in the Wild,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2018, pp. 7297–7306.

[9] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “2012 AlexNet,” Adv. Neural Inf. Process. Syst., pp. 1–9, 2012.

[10] Z. Dong, J. Qin, and Y. Wang, “Multi-stream deep networks for person to person violence detection in videos,” in Communications in Computer and Information Science, 2016, vol. 662, pp. 517–531.

[11] C. Olah, A. Mordvintsev, and L. Schubert, “Feature Visualization,” Distill, vol. 2, no. 11, p. e7, Nov. 2017.

[12] T. Hassner, “Violent-Flows – Crowd Violence Non-violence Database and benchmark,” 2014.

[13] Y. Gao, H. Liu, X. Sun, C. Wang, and Y. Liu, “Violence detection using Oriented VIolent Flows,” Image and Vision Computing, vol. 48–49. pp. 37–41, 2016.

[14] E. Bermejo Nievas, O. Deniz Suarez, G. Bueno García, and R. Sukthankar, “Violence Detection in Video Using Computer Vision Techniques,” Springer, Berlin, Heidelberg, 2011, pp. 332–339.

[15] O. Kliper-Gross, T. Hassner, and L. Wolf, “The action similarity labeling challenge,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 3, pp. 615–621, 2012.
 

The Application of Air-Coupled Impact-Echo Method to Detect Concrete Structure

Abstract
The damage that is induced by corrosion is one of the most serious problems affecting the service life for reinforced concrete structures. Corrosion-induced delamination causes the concrete bridge deck deterioration [1]. Recently, structural failures in Europe have attracted concern about post-tensioned concrete structures; voids within grouted tendon ducts, due to insufficient grout filling, significantly accelerate the corrosion of the embedded steel tendons [2]. Nondestructive evaluation (NDE) techniques that can detect, locate, and characterize delamination and duct void defects in concrete draw great interests to agencies of infrastructure management [3]. Air-coupled impact-echo is one of the successful applications for nondestructive evaluation of concrete. And this final report will describe the air-coupled impact-echo method and discuss how it applies to a concrete structure.

Get Help With Your Essay
If you need assistance with writing your essay, our professional essay writing service is here to help!
Essay Writing Service

Description of Impact-echo Method
First, let us briefly start with what is the impact-echo method. The impact-echo method is a technique used for detecting cracks inside structures. This method is based on monitoring the surface motion due to a short-duration mechanical impact. And it overcomes many barriers associated with flaw detection. We first tap an object with a hammer and the integrity of the structural member can be assessed based on whether the result is a high-pitched sound or a low-frequency sound. The method is subjective because it depends on the experience of the operator, and it is only limited to detecting near surface defects.

Figure 1
In the nondestructive testing of metals, the ultrasonic pulse-echo technique is well known to be a reliable method for locating cracks or other internal defects. An electromechanical transducer is used to produce a short pulse of stress waves which can propagate into the inspected object. Reflection of the stress pulse occurs at boundaries divided materials with different densities and different kinds of elastic properties. The transducer that also acts as a receiver will receive the reflected wave. The received signal is displayed on an oscilloscope, and the time of travel of the pulse is measured electronically. With the known speed of the ultrasonic stress wave, we can determine the distance to the reflecting interface [15].
Basic relationship
When a disturbance (stress or displacement) is suddenly applied at a point on the surface of a solid, such as through an impact, the disturbance can propagates through the solid as one of three different types of stress waves: a P-wave (In SE 263, it is called a longitudinal wave), an S-wave (In SE 263, it is called a shear wave), and an R-wave (In SE 263, it is called a Rayleigh or surface wave). As shown in Figure 1, the P-wave and S-wave propagate into the solid along spherical wave fronts. The P-wave is associated with the normal stress propagation and the S-wave is related to shear stress. Besides, the R-wave can travel away from the disturbance along the surface of object.
Figure 2 demonstrates the results of a finite-element analysis of an impact response concerning a plate [4]. This figure is a plot of the nodal displacements in the finite element mesh. In this analysis, the S-wave arrives at the bottom of the plate and the P-wave reflection is around halfway up the plate.

Figure 2
In an isotropic, infinite and elastic solid, the P-wave speed, Cp, is associated with Young’s modulus of elasticity, Poisson’s ratio and the density.
CP= E(1–v)ρ(1+v)(1–2v)
The S-wave, Cs, is below:
CS= Gρ= E2ρ(1+v)
where G is the shear modulus of elasticity.
The ratio of S-wave speed to P-wave speed is below:
CsCp=1–2v2(1–v)
Generally, with a Poisson’s ratio of 0.2 for concrete, ratio equals 0.61. The ratio of the velocity of R-wave, Cr, to the S-wave speed is given by the following approximate formula:
CrCS=0.87+1.12v1+v
Impact-echo Method
The greatest success in the real-world application of stress wave methods for flaw detection in concrete is using mechanical impact to produce the stress pulse. The impact generates a high energy pulse that could penetrate deep into the concrete. The first productive applications of impact methods occurred in geotechnical engineering to evaluate the integrity of concrete piles and caissons. The technique is known as the sonic-echo or seismic-echo methodology [ACI 228.2R]. The impact response of skinny concrete members is more complicated than the one of long slender members. The research by [4], however, led to the development of the impact-echo method, which has proven to be and well known as a powerful technique for flaw detection in relatively skinny concrete structures.

Figure 3
Figure 3 is a schematic diagram of impact-echo testing on a concrete plate with an air void below the surface. The impact on the surface produces P-waves and S-waves that travel into the plate and R-waves that travel away from the impact point. The P-waves and S-waves are reflected due to the internal defects or external boundaries. When the reflected waves return to the surface, they will cause displacements which are measured by a receiving transducer. If the transducer is placed nearby the impact point, P-wave echoes will dominate the response [4]. The right-hand side of Figure 3 shows the pattern of surface displacements that will occur. The large downward displacement at the start of the waveform is caused by the R-wave. The series of repetitive downward displacements of lower amplitude is because of the arrival of the P-wave when it experiences multiple reflections between the surface of concrete and the internal void.
Frequency Analysis
In the initial work resulting in the impact-echo method, time domain analysis is used to measure the time from the beginning of the impact to the arrival of the P-wave echo [5]. While this was doable, the whole process was time-consuming and required skills to determine the time of P-wave arrival properly. A key development resulting in the success of the impact-echo method was the utilization of frequency analysis rather than time domain analysis of the waveforms [6].

Figure 4
The principle of frequency analysis is shown in Figure 4. The P-wave generated by the impact experiences multiple reflections between the surface of concrete and the reflecting interface. The P-wave arrives at the test surface for each time, which causes a characteristic displacement. Therefore, the waveform contains a pattern that based on the round-trip travel distance of the P-waves. If the receiver is near the impact point, the round-trip travel distance is double distance of surface and interface.  As shown in Figure 4, the time interval between successive arrivals of the various reflected P-wave is the distance of wave travel divided by the wave speed. And the frequency (f) of the P-waves arrival is the inverse the time interval and has the approximate relationship below:
f= Cpp2T
where
Cpp = the P-wave speed travels the thickness of the plate,
T = the depth of the reflecting interface.
Amplitude spectrum
In frequency analysis of impact echo method, the goal is to determine the dominant frequencies in the waveform. Here, we need to use the Fourier transform technique to transform the waveform into the frequency domain [7]. The transformation produces an amplitude spectrum that shows the amplitudes of the various frequencies included in the waveform. Take plate-like structures as an example, usually, the thickness frequency is the dominant peak in the spectrum.

Figure 5
The peak frequency value in the amplitude spectrum after Fourier transform is used to determine the depth of the reflecting interface by expressing equation as below:
T= Cpp2f
Figure 5 shows the use of frequency analysis of impact-echo tests.
Figure 5(a) shows an example from a test of the amplitude spectrum over a solid portion of a 0.5 m thick concrete slab. The peak frequency is at 3.4 kHz, which with respect to multiple P-wave reflections between the top and bottom surfaces of the slab. By using the equation above, the P-wave speed in the slab we got is 3420 m/s.
Figure 5(b) shows the amplitude spectrum over a portion of the slab containing a disk-shaped void [5] [6]. The peak frequency at 7.3 kHz induced by multiple reflections between the top part of the slab and the void. The calculated depth of the void is 3420/(2 x 7300) = 0.23 m, which compares with the known distance of 0.25 m.
Air-Coupled Sensing System
Air-Coupled Sensor
A measurement microphone manufactured by PCB Inc. was used in the air-coupled impact echo tests. It has a small size, 6.3 mm diameter, broad frequency range (4-80 kHz at ±2 dB), and high sensitivity 4 mV/Pa. This sensor is well compatible for impact echo scanning because it can detect a broad range of frequencies and the small size is able to improve spatial resolution in scanning tests [16].
A special enclosure was used to support the microphone. And it provides sound insulation to block ambient noise and direct acoustic waves.
Figure 6 shows a drawing of the microphone and the insulation enclosure. The enclosure wall has an inner layer of rubber, a cylinder made in aluminum, and an outer foam layer. The foam and aluminum work at the same time to absorb and reflect most ambient noise and direct acoustic waves, in the contrast, the inner rubber layer absorbs the leaky waves from the concrete surface and inhibit the formation of resonances inside the cylindrical enclosure cavity. The microphone is inbuilt into the enclosure by crossing a hole. The microphone height is able to be easily adjusted. Experimental studies were carried out to work on the sound insulating efficiency of the enclosure. The tests were implemented on two same concrete slabs that were placed near each other, but fully isolated. The impact was applied to one slab and at the same time, the sensor monitored that same slab at a fixed spacing configuration to measure the total response impact echo plus direct acoustic wave [16].

Figure 6
As shown in Figure 7, with respect to the total response amplitude, this design of enclosure reduced the amplitude of direct acoustic waves by 40% from 50 to 10%. Figure 7a shows the signals in the situation of without insulation, the direct acoustic wave/total response amplitude ratio is 1:2. In contrast, in Figure 7b, the ratio is 1:10. Meanwhile, ambient noise amplitudes were significantly dropped [16].

Figure 7
Air-Coupled Sensing
As far as the problem of slow testing rate of mechanical wave methods, one solution for it is the application of contact-less sensing. By erasing the contact between sensors and concrete surfaces, the possibility of an automated scanning system can be considerable. The air-coupled acoustic sensors can be used for contact-less mechanical wave detection in solids. However, the intrinsic rough surface of the concrete has some limitations in laser application. Despite the huge four orders of magnitude acoustic impedance mismatch between solids and air, air-coupled ultrasonic sensing has experienced rapid development in recent decades, especially for guided wave detection in metals [8]. However, the inhomogeneous concrete limits the practical application of fully air-coupled contact-less excitation and detection ultrasonic methods [9]. The wave energy transmission is obviously increased when contact is used with an air-coupled receiving sensor. Although this system is not fully contact-less, elimination of surface coupling for the receiving sensor reduces testing time and air-coupled sensors are prone to show improved signal consistency over contact sensors [10]. Efforts to use air-coupled acoustic sensors to inspect concrete date back to 1973, when the Texas Transportation Institute in College Station, Tex. developed an automated delamination detection device called Delamatec [11]. The essential components of Delamatec are automated tappers, a strip chart recorder, and acoustic receivers. When applied over sound defect-free concrete, the obtained time domain signal is very close to zero; the signal becomes irregular whenever delamination. However, due to poor accuracy, the application of the Delamatec has been limited.
More recently air-coupled sensing for surface waves in concrete structures was proposed by Zhu and Popovics in 2001. The test results, which were proved by comprehensive theoretical analyses [12], demonstrated that directional microphones are very sensitive to leaky surface waves propagating along with concrete. Leaky surface waves exist at the boundary between a solid and arise from the propagating mechanical surface waves in the solid: The resulting wave motion at a point on the surface of the solid produce acoustic waves that leak into the surrounding fluid [13]. Subsequent studies by Zhu [12] have shown that air-coupled sensors can replace contact sensors in most surface wave measurement tests. Further, air-coupled surface wave sensing can also be applied to locate surface cracks in concrete slabs [12].
Impact-echo is well known to be effective for defect detection in concrete. However, the application of air-coupled sensing for impact-echo is more challenging than for surface waves. Air-coupled impact-echo scanning tests were implemented over two concrete slabs that contain embedded artificial defects. Test results, shown as images, identify locations and areal size of most of the defects.

Figure 8 
Testing Setup
Figure 8 is shown is the testing setup of air-coupled impact-echo. The configuration is very similar to conventional impact-echo, and the only difference is there is no contact between the sensor and the test surface. The sensor is located nearby the impact location and the distance between the sensor axial projection point on the surface and the impact point x is less than 40% of the slab thickness. In air-coupled leaky surface wave detection [12], direct acoustic waves do not break the time signal as the acoustic waves can be isolated by increasing the source-receiver spacing x. Moreover, the leaky surface wave pulse is isolated and extracted by applying a Hanning window to the time signal. However, in the impact-echo testing scheme, the sensor is installed nearby the impact location. Because of the relatively small source-receiver spacing in the impact-echo test setup, the impact source induced much acoustic noise in the received signals, which cannot be isolated and eliminated in the time domain.

Figure 9
Concrete Specimens Containing Artificial Defects
Two steel-reinforced concrete slabs were cast. The slabs are nominally 0.25 m thick with 1.5m by 2.0 m lateral dimensions. The 28-day compressive strength of the concrete is 42.3 MPa. The p-wave velocity of the mature concrete is 4,100–4,200 m/s. This results in a nominal full thickness impact-echo frequency of 7.8–8.0 kHz through previous researches [16].
Slab 1 contains two continuous embedded ducts. Each duct has three sections: fully grouted, half grouted, and un-grouted. The voids of the half-grouted and un-grouted regions are simulated with foam inserts. The diameters of both ducts are 70 mm, and the centerlines of the which are 125 mm below the surface. The same specimen also contains a surface-opening notch with linearly increasing depth. The plan view and cross-section of the slab source-receiver are shown in Figure 9 [16].

Figure 10
Slab 2 contains artificial delamination and voids of various sizes and depths. The plan view and cross-section of the slab is shown in Figure 10. Because the loading capacity of the slab is dramatically reduced by artificial defects, the slabs are reinforced in two dimensions and at two layers. The top layer of steel bars is supported by 5 steel chairs. The concrete cover thickness is 60 mm. Metal wire mesh 150mm by 150 mm was placed above each rebar layer. Artificial delamination was simulated by embedding 6 double-layer plastic sheets. Three double-layer sheets are located 60 mm below the surface sheets and three 200 mm below the top surface of bottom sheets. The actual depths of the sheets were measured in the slab form before casting concrete and are shown in Table 1 [16].

Table 1
Air-Coupled Impact-Echo for Delamination Detection
 Point Test Result
Individual air-coupled impact-echo tests were implemented over solid no defect and all defect regions on Slab 2. The goal here is to demonstrate that contact and air-coupled impact-echo results are effectively equivalent. The nine defects are labeled 1 to 9 from the left top corner to the right bottom corner Figure 10. The sensor was installed over the center of each defect where defect regions were tested [16].
The results from air-coupled impact-echo are shown in Table 1. The test locations are grouped based on the type of defects. For instance, Test Locations 1, 3, and 6 are over shallow delamination about 55 mm below the top surface. There is a defect at Point 0, where the frequency 7.8 kHz was obtained [16].
Flexural mode frequency measurement is more compatible and gets less influenced by ambient noise than that of the impact-echo resonance mode. The flexural mode frequency measured by the air-coupled sensor without the sound insulation enclosure agrees with the consequence measured by a standard contact sensor. The emitted impact sound direct acoustic waves frequency corresponds to the flexural mode frequency. This theory explains why conventional sounding methods [16].
Because the signals over shallow delamination are dominated by flexural vibration, the depth of delamination cannot be inferred directly from the dominant peak frequency. Based on the conventional impact-echo equation, the impact-echo mode frequency should be large for shallow delamination. Therefore, broader frequency contents must be investigated if depth information is to be acquired. Figure 11a and b show impact-echo signals acquired from the air-coupled sensor over a shallow delamination Defect 8. In the frequency spectrum, a peak at 33.0 kHz is seen in addition to that at 2.7 kHz, which corresponds to the flexural mode.
For exciting high frequencies, a very small ball (5 mm diameter impactor) need to be used since the maximum exciting frequency is inversely related to increasing ball size [14]. The 33.0 kHz peak frequency, however, is hard to detect with a standard contact impact-echo sensor even when a small impactor is used, as seen in Figure 11c and Figure 11d. The contact sensor detects vertical out-of-plane displacement on the concrete surface. The air-coupled sensor measures air pressure, which is equivalent to the out-of-plane velocity on the surface [12]. Displacement sensors are less sensitive to higher frequency content than velocity sensors. Velocity responses can represent satisfactory sensitivity across a broader and higher range of frequency.

Figure 11
Two-Dimensional Imaging
A two-dimensional 2D scanning test was implemented over the entire area of Slab 2 with 200 cm by 150 cm. The measurement of grid spacing is x = y = 10 cm for both directions; therefore, totally, 261 signals were acquired. None of the data were collected along the slab edges. The property of the contact-less sensor allowed enough scanning of the specimen, with a testing time of approximately 10 seconds per point. The testing efficiency would be improved in the future if an array of sensors were employed. Data were collected along parallel to the scan lines. A 2D matrix composed of the frequency of the amplitude spectrum of a signal at the highest amplitude peak frequency at each testing location is used for image construction. Figure 12 shows the 2D scan contour image of Slab 2. The image was created using the “contour” plotting function for the contour lines in MATLAB. In the color image, warm colors represent high frequencies, while cold colors represent low frequencies. The designed defect locations and areal size are demonstrated on the image with solid lines [16].
Most of the defects are identified in the image. The large and shallow delamination and voids agree well with the actual areal size. For the small defects, the image shows frequencies that are lower than the normal full-thickness frequency. This indicates the possible presence of small defects. Although the size and depth of the small defects can’t be precisely determined, they can still be different from the surrounding solid regions. Hot spots high frequency is observed over Defect 2 and 7, which indicates the existence of deep delamination. The peak frequency will shift to a lower frequency as the test point is located over the edges of the defect.
In addition to the nine designed defects, regions labeled “A” and “B” in Figure 12 show warmer colors from the surrounding solid regions. These regions show a higher frequency than the full-thickness frequency at 7.8 kHz even though there are no intended defects in these regions [16].

Figure 12
Refined Scans
Refined 2D scans were conducted over Defects 1 shallow delamination, 7, and 9 deep delaminations by using a 5 cm scan spacing. The imaged regions for Defects 1 and 7 are 40 cm by 40 cm of squares and for Defect 9 a 25 cm by 20 cm of the rectangle. The scan images are shown in Figure 13 below. The refined scans provide an improved definition of defect size, especially for the shallow delamination such as Defect 1. In Figure 13a, a clear boundary is observed between the delaminated and solid regions. The lowest flexural mode frequency is acquired over most of the central region over the defect, and the frequency increases when the test point moves to the nearby edges of the defect. In contrast, Figure 13b represents two-dimensional contour images of Defect 7 in slab 2, and Figure 13c represents two-dimensional contour images of Defect 9 in slab 2 [16].

Figure 13
Conclusion
Based on the testing and simulation we carried out above, we can get the conclusion below:

Air-coupled sensing provides a method for the effective evaluation of concrete structures through imaging. Multiple point data that are presented together in one image can provide more diagnostic information than the same data which are evaluated individually [16].
The amplitude spectra of individual air-coupled impact-echo signals are equivalent to those acquired from traditional contact impact-echo sensors. The same analysis and interpretation procedures can be applied to both cases [16].
Air-coupled sensors with broad frequency response can offer much more information about the dynamic response of concrete slabs that include shallow delamination when both the impact-echo and flexural vibration responses from the delamination are sensed [16].
In 2D impact-echo scan images, flexural resonances from shallow delamination permit precise definition of defect areal size. The depth of shallow delamination can’t be determined from the flexural resonance frequency directly. However, in contrast, the depth of deep defects can be determined directly from the impact-echo resonance frequency as measured above the center of the defect [16].
Finer scan point spacing improves the ability to define defect areal size in the created 2D image. To define it in the image precisely, scan point spacing should be less than half of the areal size of a defect. A 2 cm scan spacing is adequate to define all defects in a concrete structure [16].

References

[1] Bettigole, N. H., and Robinson, R. 1997. Bridge decks: Design, construction, rehabilitation, replacement, ASCE, New York.
[2] Martin, J., Broughton, K. J., Giannopolous, A., Hardy, M. S. A., and Forde, M. C. 2001. “Ultrasonic tomography of grouted duct post-tensioned reinforced concrete bridge beams.” NDT & E Int., 34(2), 107–113.
[3] Rens, K. L. 1998. “Recent trends in nondestructive inspections in state highway agencies.” J. Perform. Constr. Facil., 94–96.
[4] Carino, N.J., and Sansalone, M., 1984, “Pulse-Echo Method for Flaw Detection in Concrete,” Technical Note 1199, National Bureau of Standards, July.
[5] Carino, N.J., Sansalone, M., and Hsu, N.N., 1986a, “A Point Source – Point Receiver Technique for Flaw Detection in Concrete,” Journal of the American Concrete Institute, Vol. 83, No. 2, April, pp. 199-208.
[6] Carino, N.J., Sansalone, M., and Hsu, N.N., 1986b, “Flaw Detection in Concrete by Frequency Spectrum Analysis of Impact-Echo Waveforms,” in International Advances in Nondestructive Testing, Ed. W.J. McGonnagle, Gordon & Breach Science Publishers, New York, pp. 117- 146.
[7] Bracewell, R., 1978, The Fourier Transform and its Applications, 2nd Ed., McGraw-Hill Book Co., 444 p.
[8] Wright, W., and Hutchins, D. 1999. “Air-coupled ultrasonic testing of metals using broadband pulses in through-transmission.” Ultrasonics, 37(1), 19–22.
[9] Cetrangolo, G. P., and Popovics, J. S. 2006. “The measurement of P-wave velocity through concrete using air-coupled transducers.” Proc., NDE Conf. on Civil Engineering, I. Al-Qadi and G. Washer, eds., American Society for Nondestructive Testing, Columbus, Ohio, 180–187
[10] Berriman, J., Purnell, P., Hutchins, D. A., and Neild, A. 2005. “Humid-ity and aggregate correction factors for air-coupled ultrasonic evaluation of concrete.” Ultrasonics, 43(4), 211–217.
[11] Moore, W. 1973. “Detection of bridge deterioration.” Highw. Res. Rec., 451, 53–61.
[12] Zhu, J., Popovics, J. S., and Schubert, F. 2004. “Leaky Rayleigh and Scholte waves at the fluid-solid interface subjected to transient point loading.” J. Acoust. Soc. Am., 116(4), 2101–2110.
[13] Viktorov, I. A. 1967. Rayleigh and Lamb waves, Plenum, New York.
[14] Sansalone, M. J., and Streett, W. B. 1997. Impact-echo—Nondestructive evaluation for concrete and masonry, Bullbirer, Ithaca, N.Y.
[15] Carino, N.J., 2001,Washington, D.C., American Society of Civil Engineers, Reston, Virginia, Peter C. Chang, Editor, 2001. 18 p.
[16] Zhu, J., Popovics, J. S.,2007, Imaging Concrete Structures Using Air-Coupled Impact-Echo, J. Eng. Mech., 2007, 133(6): 628-640

 

Using Machine Learning to Detect Anomalies in Embedded Networks in Heavy Vehicles

Abstract
Consumer vehicles have been demonstrated to be insecure; the addition of electronics to monitor and control vehicle capacities has included complexity resulting in security basic vulnerabilities. Although academic research has shown vulnerabilities in consumer automobiles long back, the general public has only recently been made aware of such vulnerabilities. Modern Automobiles have more than 70 electronic control units (ECU’s).
This paper proposes to use machine learning to support domain-experts by avoiding them from contemplating irrelevant data and rather pointing them to the important parts within the recordings. The basic idea is to learn the typical behavior from the accessible timing Analysis and then to independently identify unexpected deviations and report them as anomalies. Our proposed model’s main motive is to try to find the better architecture model and Hyperparameters for the model. We used LSTM auto encoder technique to find sophisticated anomalies with varied hyper-parameters.
Keywords: Anomaly Detection. SAE-J1939. Heavy Vehicle Security
1. INTRODUCTION
Vehicles are an integral part of our life and automobile technology has evolved over the past century to address our growing needs. Earlier, a driver had to manually control various functions in a vehicle, but now a lot of these tasks have been delegated to various micro-controllers and electronic chips attached to the vehicle [8]. Modern vehicles are a collection of various Electronic Control Units (ECU), Sensor and Actuators. These ECU’s get input from different sensors and perform various mechanical actions using actuators. CAN bus is a broadcast bus, where each connected ECU pushes broadcast messages on it. These broadcast CAN messages don’t have explicit information about which ECU generated the message and any message available on the network will considered as ‘trusted’ by default. As a result if any malicious message is introduced into the network, either by a malicious ECU or an attacker, will also be considered as valid and can result in abnormal behavior Apart from doing their own functions, ECU’s must communicate between each other so as to efficiently perform their functions [5].

Get Help With Your Essay
If you need assistance with writing your essay, our professional essay writing service is here to help!
Essay Writing Service

The past studies analyzed multiple attack vectors in vehicles and showed that Electronic Control Units (ECUs) could be compromised. A compromised ECU can be used by an attacker to inject malicious messages into the in-vehicle network through a physical connection [9]. Heavy vehicles use a standardized protocol known as SAE J1939 implemented over a Controller Area Network (CAN) bus using which the ECUs communicate with each other. The use of standardized protocols makes heavy vehicles susceptible to attacks.
Two different countermeasures have been introduced against these attacks: proactive and reactive. Proactive mechanism focuses on improving protocols and they are not fool proof but can be remarkably effective. There have been techniques proposed to include message authentication on the protocol. Reactive mechanisms detect an attack or an impending attack and reduce its impact on the victim’s vehicle at the earliest and provide a response mechanism to either block or alert other systems [6].
The uses of SAE-J1939 makes it possible to convert raw transmitted messages on the CAN bus to specific parameters of the vehicle. Thus, we define a machine learning model based on low-level vehicular parameters. While each message contains information about the current state of the vehicle, it does not give any information about the previous state. To solve this limitation, we added the history of previous values to each parameter value to leverage the learning model. In addition, some statistical derivative features have been added to give even deeper clues to the model [8].
A vehicle’s parameters are categorized in particular groups in the SAE-J1939 based on, for example, frequency and sender. Thus, we created multiple models based on each group of parameter, referred to as Parameter Group Number (PGN) in the standard. The learning algorithms create a behavioral profile for each PGN that will be used later to compare with its current behavior to detect any deviation from the regular pattern. We used a wide range of learning algorithms to train models and studied their performance [8].
The proposed approach integrates four modules to detect anomalies. BusSniffer connects to the bus and sniffs the messages. Message Decoder gets messages from BusSniffer and converts them into raw messages that characterize the vehicle’s parameters. AttackDetector compares the current state with the appropriately trained model and triggers the AlarmGenerator if a threat exists. Based on these modules we can generate real time alarms and thereby providing security to the protocol [6].

Fig 1: Example SPN layout for the “Engine Temperature” PGN [7]
The rest of the paper is organized as follows:
Section 2 – Background which contains CAN and SAE J1939 protocols and the defense mechanisms.
Section 3 – Adversary Threat Model of modern attacks.
Section 4 – Features that we use.
Section 5 – Detection Mechanism Architecture.
Section 6 – Building of Machine Learning Model.
Section 7 – Conclusion and Future work.
2. BACKGROUND
In this Section, we discusses about CAN and SAE J1939 protocols and how they are introduces and evaluated. Also discusses about the defense mechanisms.
Controller Area Network (CAN) is a serial organized innovation that was initially outlined for the car industry, particularly for European cars, but has also become a prevalent bus in industrial automation as well as other applications. The CAN bus is basically utilized in embedded systems, and as its title suggests, is a network innovation that gives fast communication among microcontrollers up to real-time requirements. CAN 1.0 was introduced in the time that neither Internet nor any evidences of virus were seen and security is not at all a concern at that time. This indicated that CAN protocol cannot address security concerns [5].
SAE-J1939 Standard: SAE J1939 is the open standard for networking and communication in the commercial vehicle sector. There are a number of measures which are determined from SAE J1939. These guidelines utilize the fundamental portrayal of J1939 and regularly contrast as it were in their data definition and adaptations of the physical layer.SAE-J1939 characterizes five layers within the seven-layer OSI network model including the CAN ISO 11898 specification and employments as it were expanded outlines with a 29-bit identifier for the physical and data-link layers [8]. Each PDU in the SAE-J1939 protocol consists of seven fields: priority (P), extended data page, data page (DP), PDU format (PF), PDU specific (PS) (which can be a destination address, group extension, or proprietary), source address (SA), and data field. There is also a reserved field Reserved(R) with one bit length for further usage [2].

Fig 2: Architecture of CAN and SAE J1939 protocols.
Proactive mechanism: Proactive mechanism focuses on improving protocols but the CAN and SAE-J1939 protocols do not support any authentications which lead to wide attacks .However even with an authentication mechanism on the CAN bus the maximum payload length would be just 8 bytes so the space for MAC (Message Authentication Code) is so limited [6].
Reactive Mechanism: Reactive mechanisms distinguish an attack or an impending attack and diminish its effect on the victim’s vehicle at the earliest and provide a response mechanism to either block the attack or alert other frameworks [6].
The use of different machine learning algorithms came into existence in detecting anomalies through packets and packet sequences. Usage of Long Short Term Memory (LSTM) came to consideration that is used for the sequence of inputs for the datasets. One layer of LSTM has as many cells as the time steps. The objective of the Autoencoder network in is to reconstruct the input and classify the poorly reconstructed samples as a rare event.
3. THREAT MODEL
Attackers can easily compromise ECU’s and thereby exploiting new vulnerabilities. There’s more motivating force for an enemy to attack the heavy vehicle industry due to the size of the vehicles and the assortment of goods they carry. Our adversary can be anybody who might stand to make a profit on controlling the vehicles, be it from hijacking their merchandise, adversely controlling a competition’s fleet, extorting fleet proprietors and drivers, or offering their tools and administrations on the black market. Another sort of adversary we consider is one who wishes to cause the most harm and harm as possible, such as a terrorists. We expect that our adversary has the ability to transmit selfassertive messages on the vehicle’s J1939 bus. This is often most promptly accomplished with physical access to the vehicle through the OBD port [5]. We assume that the adversary will receive messages on the CAN bus and can generate SAE J1939 compatible messages with the frequency including the data and priority [9]. Attackers will take control over the Message priority and can block the messages with lowest priorities on the bus. This affects the functions and integrity of the system in the exploitation.
On the other hand, a more sophisticated attacker could inject malware into other ECUs. These attacks will reflect on the CAN level and apply to both regular and heavy vehicles. The most common attack against the CAN network is a DoS attack [6]. In this attack, the adversary will send unauthorized messages with the most elevated priority and frequency to dominate the bus. Thus, sending or accepting messages will be deferred or indeed inconceivable. In a different attack, an adversary may monitor the CAN bus and target a specific activity of the vehicle. At whatever point the adversary sees a message related to that specific activity, it sends a counter message to make the past action ineffective. In this case, an attacker can either dominate the initial engine’s ECUs with a higher priority message or can send an incorrect value for a particular parameter after seeing it on the bus.
There’s not any attack data freely accessible to be utilized as a benchmark. So, we simulated modern attack messages and injected them into the logged file to check whether our detection component could find them [7]. During our proposed attack, we malevolently changed the vehicle’s parameters (such as current speed) multiple times.
4. DEFENCE MECHANSIM
In our proposed model, the performance relies on the choice of features and how to implement them. We define three features in our paper: SPN values, History values and Derivative features.
SPN Values: Features in SPN Values are obtained from deciphering messages on the CAN bus. We convert the raw messages to the SPN values [6].
History of values: The value of each SPN depends on both the current vehicle’s parameters and their past values. The classifier would need to use past samples to create a more exact choice [6]. Towards the conclusion, we include past SPN values of each vector to overcome this challenge. As such, each vector will presently have values of the current state and will moreover include the final detailed values for each SPN.
Derivative Features: To give more detailed insight, we add multiple derivative features to the vector. We also added average, standard deviation and slope to the last n values. We add history for these features as well. The new derivative features will help classifiers to get more precise predictions.

Fig 3: SPN and PGN Data bit fields
5. Detection Mechanism Architecture
The proposed architecture consists of four separate modules: BusSniffer, Message Decoder, AttackDetector, and AlarmGenerator.
BusSniffer interfaces to the CAN bus using an access point like the OBD-II port. This port connects specifically to the CAN bus and generates all transmitted messages on the CAN bus.
MessageDecoder utilizes the SAE-J1939 standard to convert the raw messages to the SPN values thereby creating an initial vector of the vehicle’s parameters. This module includes other meta-data fields including time-stamp, length of the data field, source address, destination address, and previously defined features such as derivative features and history of feature values.
AttackDetector consists of two phases: Training and Detecting. The training phase requires preparing a dataset of regular and abnormal messages for every PGN. Multiple classifiers can be trained on the dataset, and the classifier that performs the best will be used. The training phase may take a long time; the trained classifiers can be used countless times without the need to retrain them.
In the detection phase, whenever a new vector comes in, the AttackDetector fetches the PGN value from the vector and sends it to the designated classifier object. The classifier then tests whether it is a normal vector. If the classifier detects an abnormal message, it will produce the AlarmGenerator module. AlarmGenerator is responsible for preparing alarm messages using SAE-J1939 and transmits it over the CAN bus. The message will be generated in the form of a Broadcast message, and all connected nodes will be aware of this abnormal situation. This can also include turning on a warning light on the dashboard to notify the driver [6].

Fig 4: Architecture of proposed detection mechanism
6. Building Machine Learning Model
Building of our experiment takes place in five different phases to get the desired outcome. They are:

Gathering the Datasets
Data Pre-processing
Building the Machine Learning Model
Building the Architecture
Evaluations

6.1 Gathering the Datasets
We used several CAN bus log messages that were generated previously. We required a lot of data with many messages in the log and we also require PGNs. A Parameter Group Number (PGN) is a part of the 29-bit identifier sent with each log message. The PGN is a combination of the Reserved bit, the data page bit, the PDU Format (PF) and PDU Specific (PS). We also need SPNs in our project. A Suspect Parameter Number (SPN) is a number that has been assigned to a specific parameter within a parameter group [9]. SPNs that have similar characteristics will be grouped into PGNs. Since the log messages had more PGNs we used more instances for the training and testing phases. Developing profiles of PGNs by using machine learning techniques can be generated by ECUs.
6.2 Data Pre-processing
Data pre-processing takes place in three different phases. They are Training Datasets, Validation sets and Testing.
Training datasets: The sample of data that we use to fit into the model is the training dataset. We train the sample so as to pre-process the data that is required for our model.
Validation set: We use the validation set to fine tune the model hyper-parameters.
Testing datasets: The testing data set is the final data that we use to get the desired output in the model which considers both training set and validation set [1].

Fig 5: Training set, Validation set, Test set Process
6.3 Building the Machine Learning Model
In building our model we use LSTMs. Long Short term memory (LSTM) is an artificial RNN used in the field of deep learning which are capable of learning long-term dependencies. Unlike others LSTM has feedback connections. LSTMs are used in image, speech and video sequence data. Our proposed model is a sequential model so we chose LSTM. They can remember things for a long duration of time. The LSTM have the capacity to expel or include data to the cell state, carefully controlled by structures called gates [1].
Our Proposed Model has sequential data, so we used Encoder-Decoder LSTM architecture. e. Our method uses a multi layered Long Short-Term Memory (LSTM) to phrase the input to a vector and then deep LSTM to decode the output from the vector. The core of our experiments involved training a large deep LSTM auto-encoder. The LSTM is capable of solving long term dependencies but it works efficiently when the source is reversed [1]. The LSTM auto encoder first compresses the input data and then uses repeat vector layer. The final output layer gets back the reconstructed input data. LSTMs trained on reversed source data did much superior on long sentences than LSTMs trained on the raw data. We found that LSTM models are easy to train with more effective results.
Our LSTM decides what information we’re going to put away from the cell state. This decision is made by a sigmoid layer called the “forget gate layer.” It looks at ht−1 and xt, and outputs a number between 0 and 1 for each number in the cell state Ct−1 [2] (htt).
6.4 Building the Architecture
In our architecture we use 3 LSTMs, one input layer and one output layer. We use sigmoid function in the LSTMs specifically because it is used as the gating function for the 3 gates (in, out, forget), since its outputs are always a value between 0 and 1, it can either let no flow through or complete the flow of information throughout the gates. The activation function we use is the ReLU activation function. ReLU stands for rectified linear unit.

Fig 6: LSTM Layers and their functions.
Mathematically, it is defined as y = max(0, x) [10]. We use this activation function because it allows our model to run or train easily.

Fig 7: Activation Function of ReLU
In the compilation, we use Loss function and optimizers. The loss function use used is Mean Square Error (mse). The groups of functions that are minimized are } called “loss functions”. A loss function is a degree of how great a prediction model does in terms of being able to foresee the anticipated outcome. It depends on a number of variables counting the presence of outliers, choice of machine learning algorithm, time effectiveness of gradient descent, ease of finding the derivatives and certainty of predictions. MSE is the sum of squared distances between our target variable and predicted values.

Fig 8: Loss function of MSE
The optimizer we used in the model is ‘Adam’. Adam is an adaptive learning rate method; it computes learning rates for different parameters. Adam uses estimations of first and second moments of gradient to adjust the learning rate for each weight of the neural network. Adam is an optimization algorithm that can be used to update network weights in training data. Using of Adam makes the model to present results in a quick an effective way.
6.5 Evaluations
In This phase, we start fitting the data we collected. The challenge here is we should not over fit the model so we use Hyper-parameter Tuning. Hyper-parameter tuning is nothing but setting a value to the absolute learning process evaluation module when it begins. Hyper-parameters are passed in as arguments to the constructor of the model classes. With this, the values of other parameters are learned. Hyperparameter Tuning finds a tuple of hyperparameters that yields an ideal model which minimizes a predefined loss function on given autonomous data. Too many epochs can lead to overfitting of the training dataset, so we used Early Stopping function.
The number of epochs and the batch size determines the accuracy and performance of the model. So we carefully adjusted the batch sizes and epochs accordingly. We used limited epochs and the with sophisticated batch sizes. In our Model, we considered min-delta because of the overfitting problem.
For each dataset that includes several PGNs, we trained multiple datasets for each PGN given in the model. With the help of LSTM auto-encoder, it is easy to remember the past data which saves a lot of time in evaluation and the accuracy is also increased. With the help of the data we considered in various aspects became easy to find minor anomalies and the security bleaches are covered properly. The rate of false positives is significantly very low in our model which benefits the accuracy and consistency. LSTM auto-encoder can tests many kinds of datasets and parameters that can contribute towards the present machine learning scenario.
7. CONCLUSION AND FUTURE WORK
In this work, we appeared that a large deep LSTM auto encoder with a limited datasets can beat a standard SMT-based framework whose results are much more diverse and approximate. The success of our simple LSTM-based approach on the sequential data provided confirmations that it can be used to get good outputs with other sequence learning problems, provided that they have enough data to train with.
The spotting of normal behavior of devices is an important step in finding anomalies in heavy vehicles. With the results we got there is still a lot diverse modifications that should be implemented to get much better results and this is possible by training and testing different kinds of datasets in all kinds of aspects. We should try with different possibilities and also with fine tuning of different hyper-parameters. Usage of LSTMs made our experiment succeed in a different level and there’s lot of work to be inherited in the usage to get many impossible tasks to come possible.
It is sensible to expect that with more time many adversaries could make an indeed more sophisticated attack. With Bluetooth, cellular, and Wi-Fi, advanced trucks are getting to be much more connected to the exterior world, which present new attack vectors. So, I suggest these ideas are to be implemented effectively in order to stop huge attacks on the heavy vehicles securities.
REFERENCES
[1] Ilya Sutskever, O. V. (n.d.). Sequence to Sequence Learning with Neural Networks.
[2] Kyunghyun Cho, B. v. (n.d.). Learning Phrase Representations using RNN EncoderDecoder for Statistical Machine Translation., (p. 11).
[3] Macher, G. M. (n.d.). Integrated Safety and Security Development in the Automotive Domain.
[4] SAE J1939, Digital Annex, 201. (n.d.).
[5] Sandeep Nair Narayanan, S. M. (n.d.). OBD_SecureAlert: An Anomaly Detection System for Vehicles.
[6] Shirazi, H. (n.d.). Using Machine Learning to Detect Anomalies in Embedded Networks In Heavy Vehicles.
[7] Theissler, A. (2014). Anomaly detection in recordings from in-vehicle networks.
[8] Yelizaveta Burakova, B. H. (n.d.). Truck Hacking: An Experimental Analysis of the SAE J1939 Standard.
[9] Zhang, M. &. (2017). afeDrive: Online Driving Anomaly Detection From Large-Scale Vehicle Data.
[10] https://machinelearningmastery.com/rectifiedlinear-activation-function-for-deep-learning-neuralnetworks/