6 Types of Neural Networks Every Data Scientist Must Know

6 Essential Types of Neural Networks · Perceptron · Multi-layer Perceptron · Convolutional Neural Networks · Recurrent Neural Networks · Long Short ... GetstartedOpeninappSigninGetstartedFollow626KFollowers·Editors'PicksFeaturesDeepDivesGrowContributeAboutGetstartedOpeninappPhotobyMichaelDziedziconUnsplashDEEPLEARNING,NEURALNETWORKS6TypesofNeuralNetworksEveryDataScientistMustKnowThemostcommontypesofNeuralNetworksandtheirapplicationsRamyaVidiyalaDec17,2020·7minreadNeuralnetworksarerobustdeeplearningmodelscapableofsynthesizinglargeamountsofdatainseconds.Therearemanydifferenttypesofneuralnetworks,andtheyhelpusinavarietyofeverydaytasksfromrecommendingmoviesormusictohelpingusbuygroceriesonline.Similartothewayairplaneswereinspiredbybirds,neuralnetworks(NNs)areinspiredbybiologicalneuralnetworks.Thoughtheprinciplesarethesame,theprocessandthestructurescanbeverydifferent.Thisisastrueforbirdsandplanesasitisforbiologicalneuralnetworksanddeeplearningneuralnetworks.Tohelpputitintoperspective,let’slookbrieflyatthebiologicalneuronstructure.Figure1showstheanatomyofasingleneuron.Thecentralpartiscalledthecellbody,wherethenucleusresides.Variousconnectionspassthestimulustothecellbody,calleddendrites,andafewconnectionssendtheoutputtotheotherneuronscalledaxons.Thethicknessofthedendritesandaxonsimpliesthepowerofthestimulus.Manyneuronswithvariouscellbodiesarestackedupandformabiologicalneuralnetwork.Figure1:AnatomyofSingleNeuron(Imagebyauthor)Thissamestructureisvisibleindeeplearningneuralnetworks.Theinputispassedthroughanactivationfunction(similartothenucleus)withweightededges(similartodendrites).Thegeneratedoutputcanbepassedtoanotheractivationfunction.Manyoftheseactivationfunctionscanbestackedup,andeachoftheseiscalledalayer.Apartfromtheinputlayerandtheoutputlayer,therearemanylayersintheinteriorsofaneuralnetwork,andthesearecalledhiddenlayers.6EssentialTypesofNeuralNetworksNowthatwehaveapictureofhowneuralnetworkswork,let’slookatthevarioustypesandfunctionsoftheneuralnetworksusedindeeplearning.Notethateachtypeofartificialneuralnetworkistailoredtocertaintasks.We’lllookatthemostcommontypesofneuralnetworks,listedbelow:PerceptronMulti-layerPerceptronConvolutionalNeuralNetworksRecurrentNeuralNetworksLongShortTermMemoryNetworksGenerativeAdversarialNetworks1.PerceptronPerceptronisthesimplestneuralnetworkstructure.Thismodel,whichisalsoknownasasingle-layerneuralnetwork,containsonlytwolayers:TheInputLayerTheOutputLayerTherearenohiddenlayershere.Perceptrontakesinputandcalculatestheweightedinputforeachinputnode.Thisweightedinputispassedthroughanactivationfunctiontogeneratetheoutput.Figure2:Singleneuronneuralnetwork(Imagebyauthor)Duetothesimplearchitecture,itcannotbeusedforcomplextasks.ThisnetworkisinsteadusedforLogicGateslikeAND,OR,orXOR.ApplicationsPerceptronsareusedinlinearorbinarymodelclassification.Theyarealsousedintheformationofmultilayerperceptrons,whichwe’lllookatnext.2.Multi-layerPerceptronMultilayerperceptrons(MLPs),orfeedforwardneuralnetworks,usuallymeanfullyconnectednetworks.Inotherwords,eachneuroninonelayerisconnectedtoallneuronsintheadjacentlayers.Hence,anMLPhashigherprocessingpowerthanaperceptron.However,the“fully-connectedness”ofthesenetworksmakesthempronetooverfittingdata.Typicalwaystoreduceoverfittingincludeearlystopping,addingdropoutlayers,andaddingregularizationterms.Figure3:ArchitectureofMulti-layeredperceptron(Imagebyauthor)ApplicationsMLPsarewidelyusedinavarietyofareas.They’recommonindatacompressionforsocialnetworks,speechrecognitionandhand-writtencharacterrecognitionsystems,computervisionapplications,anddatapredictionsystems.3.ConvolutionalNeuralNetworksHumansidentifyobjectsusingneuronsintheeyeswhichdetectedges,shapes,depth,andmotion.Oneofthemostimportanttypesofneuralnetworksincomputervision,convolutionalneuralnetworks(CNNs)areinspiredbythevisualcortexoftheeyesandareusedforvisualtaskslikeobjectdetection.TheconvolutionlayerofaCNNiswhatsetsitapartfromotherneuralnetworks.Thislayerperformsdotproduct,whichiscomponent-wisemultiplicationfollowedbyaddition.IntheinitialphasesofaCNN,thefiltersarerandomizedanddonotprovideanyusefulresults.Usingalossfunction,thefiltersareadjustedandovermultipleiterations,thenetworkgetsbetteratachievingitstask,suchasdetectingobjectedges,forexample.Thoughtheyoftenrequirealargeamountoftrainingdata,CNNsarewidelyapplicabletoawiderangeofimageandevenlanguagetasks.Figure4:Convolutionoperationusingafilter(Imagebyauthor)ApplicationsBecauseCNNswereinspiredbythevisualcortex,theyarewidelyusedforapplicationsthatinvolvetheapplicationofcomputervision.Theseapplicationsincludefacialrecognition,facedetection,objectrecognition,handwrittenletterrecognition,andthedetectionoftumorsinmedicaldiagnosis.4.RecurrentNeuralNetworksAbookoftenconsistsofasequenceofchapters.Whenwereadaparticularchapter,wedon’ttrytounderstanditinisolation,butratherinconnectionwithpreviouschapters.Similarly,justlikenaturalneuralnetworks,machinelearningmodelsneedtounderstandatextbyutilizingalready-learnedtext.Intraditionalmachinelearningmodels,thisisimpossiblebecausewecannotstoreamodel’spreviousstages.However,recurrentNeuralNetworks(commonlycalledRNN)areatypeofneuralnetworkthatcandothisforus,makingthemusefulforapplicationsthatrequiretheuseofpastdata.Let’stakeacloserlookatRNNsbelow.Figure5:WorkingofabasicRNN(Imagebyauthor)Recurrentneuralnetworksarenetworksdesignedtointerprettemporalorsequentialinformation.RNNsuseotherdatapointsinasequencetomakebetterpredictions.Theydothisbytakingininputandreusingtheactivationsofpreviousnodesorlaternodesinthesequencetoinfluencetheoutput.AnRNNhasarepeatingmodulethattakesinputfromthepreviousstageandgivesitsoutputasinputtothenextstage.ApplicationsRNNsarecommonlyusedinconnected-sequenceapplicationssuchastimeseriesforecasting,signalprocessing,andhandwrittencharacterrecognition.Also,RNNsarewidelyusedinmusicgeneration,imagecaptioning,andpredictingstockmarketfluctuations.5.LongShort-TermMemoryNetworksInRNNs,wecanonlyretaininformationfromthemostrecentstage.Butforaproblemlikealanguagetranslation,weneedmuchmoreretention.That’swhereLSTMnetworkscomeintothepicture.Tolearnlong-termdependencies,ourneuralnetworkneedsmemorizationpower.LSTMsareaspecialcaseofRNNswhichcandothat.Theyhavethesamechain-likestructureasRNNs,butwithadifferentrepeatingmodulestructure.Thisrepeatingmodulestructureallowsthenetworktoretainamuchlargeramountofpreviousstagevalues.Figure6:HowanLSTMnetworklooks(Imagebyauthor)ApplicationsI’vealreadymentionedhowpowerfulLSTMnetworksareforlanguagetranslationsystems,buttheyhaveawiderangeofapplications.Someoftheseapplicationsincludesequence-to-sequencemodelingtaskslikeanomalydetection,speechrecognition,textsummarization,andvideoclassification.6.GenerativeAdversarialNetworksGiventrainingdata,GenerativeAdversarialNetworks(orsimply,GANs)learntogeneratenewdatawiththesamestatisticsasthetrainingdata.Forexample,ifwetrainaGANmodelonphotographs,thenatrainedmodelwillbeabletogeneratenewphotographsthatlooksimilartotheinputphotographs.AGANcontainstwoparts:ageneratorandadiscriminator.Thegeneratormodelcreatesnewdatawhilethediscriminatortriestodeterminerealdatafromgenerateddata.Asthegeneratoranddiscriminatorgetbetterattheirrespectivejobs,thegenerateddataimprovesasaresult,untilitis(ideally)nearlyidenticalinqualitytothetrainingdata.Thinkoftherelationshipasthatofcopsandrobbers.Botharealwaystryingtooutsmarttheother;therobberstosteal,andthepolicetocatchtherobbers.Figure7:ArchitectureofGenerativeAdversarialNetworks(Imagebyauthor)Usingthegenerator,wefirstcreaterandomnoisesamplesandpassthemthroughthediscriminator.Thediscriminatorcaneasilydifferentiatebetweenthetwotypesofdata,soweadjustthegeneratormodelandtrainagain.Astheiterationsincrease,thegeneratormodelcreatesdatathatisindistinguishablefromthetrainingdata.ApplicationsGANsarecommonlyusedtocreatecartoonimagesorfacesforgamingandanimatedmovies.Also,GANscanhelpgeneratesyntheticdatafromasmallamountofdatatohelpimprovemachinelearningmodels.GANsarealsoapopularchoiceforartistslookingtousemachinelearningmodelstoexpandtheirexpression.SummaryInthisarticle,welookedattherelationshipbetweenneuralnetworksandbiologicalneuralnetworks,thenjumpedintoanumberofneuralnetworksandhowtheywork.Thisarticleisageneralguidetoneuralnetworkconcepts,butthedomainisevolvingconstantly.Withtheseconceptsinhand,Ihopeitwillbeeasiertounderstandandexploretheworkingsofothertypesofneuralnetworks,andthosecurrentlybeingdeveloped.Iencourageyoutoexploremoreinthisdomainbecausethereisawiderangeofincredibleapplicationsandlotsofongoingresearch.Thanksforreading!Thisarticlewasoriginallypostedhere.Iamgoingtowritemorebeginner-friendlypostsinthefuturetoo.FollowmeuponMediumtobeinformedaboutthem.IwelcomefeedbackandcanbereachedoutonTwitterramya_vidiyalaandLinkedInRamyaVidiyala.Happylearning!RamyaVidiyalaInterestedincomputersandmachinelearning.Likestowriteaboutit|https://www.linkedin.com/in/ramya-vidiyala/Follow210210 210DataScienceMachineLearningEducationArtificialIntelligenceTechnologyMorefromTowardsDataScienceFollowYourhomefordatascience.AMediumpublicationsharingconcepts,ideasandcodes.ReadmorefromTowardsDataScienceMoreFromMediumMLmodelonadockercontainerAsthaVermaRegularPyTorchTensorfunctionswithirregularOutcomesSouptikMajumderHands-onBigDataAnalysisonGCPUsingAIPlatformNotebooksMPLinGoogleCloud-CommunityTrainaTensorFlowModelinAmazonSageMakerJunM.inTowardsDataScienceTrainYOLOv4withSpecificCategoriesofImagesinMSCOCOdatasetHenryYuBasicsofLinearRegression😃AnkurSaikiaImageInpainting — ObjectRemovalandLossRecoveryfromImagesusingDeepLearning.BharatAhujaEmojineeringPart1:MachineLearningforEmojiTrendswikitopiya