Types of Neural Networks and Definition of ... - Great Learning

The three most important types of neural networks are: Artificial Neural Networks (ANN); Convolution Neural Networks (CNN), and Recurrent Neural ... AllCourses DataScienceCourses Master’sinDataScience M.TechinDataScience(Bangalore) DataScienceSpecialization PGinBusinessAnalytics DecisionMakingUsingDataScience DataScienceandMachineLearning ArtificialIntelligenceCourses PGP–ArtificialIntelligenceandMachineLearning(Online) PGPinArtificialIntelligenceandMachineLearning(Classroom) PGP–ArtificialIntelligenceforLeaders PGinMachineLearning PGDiplomainComputerScienceandAI–IIITDelhi AIforHighSchoolStudents ManagementCourses ExecutivePGinManagement MBAinDigitalMarketing&DataSciencefromJAIN OnlineMBAinShivNadar BachelorofBusinessfromDeakinUniversity BBADegreeFromJainUniversity ExecutiveMBAfromPES NUSFutureLeadersProgram PGinProductManagementandAnalytics BachelorofBusinessAnalyticsFromDeakin CloudComputingCourses PGPinCloudComputingCourse AdvancedSoftwareEngineeringCourseIITMadras CyberSecurityCourses StanfordAdvancedCyberSecurityProgram DigitalMarketing&SalesCourses PGPinStrategicDigitalMarketingCourse AdvancedCertificateinStrategicDigitalMarketing AdvancedCertificatePrograminSales SoftwareEngineeringCourses SoftwareDevelopmentIITRoorkee JainOnlineMCA AdvancedSoftwareEngineeringIITMadras DesignThinkingCourses StanfordExecutiveDesignThinking PGinUserExperienceDesignIIT-Bombay FreeCourses DataScienceFreeCourses DataScienceFoundations ProbabilityforDataScience LinearProgrammingforDataScience IntroductiontoR DataVisualizationusingTableau MachineLearningFreeCourses BasicsofMachineLearning StatisticsforMachineLearning FakeNewsDetectionusingMachineLearning ProductCategorizationusingMachineLearning PythonforMachineLearning ArtificialIntelligenceFreeCourses IntroductiontoArtificialIntelligence ArtificialIntelligenceProjects IntroductiontoDeepLearning AIforLeaders IntroductiontoNaturalLanguageProcessing CyberSecurityFreeCourses IntroductiontoCyberSecurity TypesofCyberSecurity CyberSecurityThreats CyberSecurityForLeadership CyberForensics CloudComputingFreeCourses CloudFoundations CloudComputingServiceModels BestCloudCertifications CloudComputingArchitecture ApplicationsofCloudComputing BigDataFreeCourses IntroductiontoBigDataandHadoop MasteringBigDataAnalytics HadoopMapReduce IntroductiontoApacheHive SparkBasics MarketingFreeCourses IntroductiontoDigitalMarketing SEOforBeginners ContentMarketingBasics SearchEngineMarketing AffiliateMarketing EmailMarketing InterviewQuestions PythonInterviewQuestions JavaInterviewQuestions SQLInterviewQuestions MachineLearningInterviewQuestions SeleniumInterviewQuestions NLPInterviewQuestions C++InterviewQuestions AWSInterviewQuestions RPAInterviewQuestions CloudComputingInterviewQuestions Career CareerTopics CareerinAI Careerinpython CareerinCyberSecurity CareerinDigitalMarketer CareerinCloudArchitect CareerinBusinessAnalytics CareerinEthicalHacking Salary DataScientistSalary PythonDeveloperSalary CyberSecuritySalary DataAnalystSalary BusinessAnalystSalary MachineLearningSalary BlockchainDeveloperSalary DigitalMarketingSalary DevOpsSalary Skills DataScientistSkills BigDataAnalyticsSkills CloudArchitectSkills ManagementSkills CloudEngineerSkills AdministrativeSkills DigitalMarketingSkills DevOpsEngineerSkills Resume WebDeveloperResume BusinessAnalystResume DataScienceResume JavaDeveloperResume DigitalMarketingResume PythonDeveloperResume DevOpsEngineerResume ArtificialIntelligenceResume MachineLearningResume Jobs HighestPayingJobsintheWorld HighestPayingFinanceJobs HighestPayingFreelancingJobs HighestPayingITJobs Non–TechnicalJobs OtherResources ArtificialIntelligence MachineLearning PythonProgramming DataScience CloudComputing CyberSecurity BusinessManagement VisualStories Search AllCourses DataScienceCourses Master’sinDataScience M.TechinDataScience(Bangalore) DataScienceSpecialization PGinBusinessAnalytics DecisionMakingUsingDataScience DataScienceandMachineLearning ArtificialIntelligenceCourses PGP–ArtificialIntelligenceandMachineLearning(Online) PGPinArtificialIntelligenceandMachineLearning(Classroom) PGP–ArtificialIntelligenceforLeaders PGinMachineLearning PGDiplomainComputerScienceandAI–IIITDelhi AIforHighSchoolStudents ManagementCourses ExecutivePGinManagement MBAinDigitalMarketing&DataSciencefromJAIN OnlineMBAinShivNadar BachelorofBusinessfromDeakinUniversity BBADegreeFromJainUniversity ExecutiveMBAfromPES NUSFutureLeadersProgram PGinProductManagementandAnalytics BachelorofBusinessAnalyticsFromDeakin CloudComputingCourses PGPinCloudComputingCourse AdvancedSoftwareEngineeringCourseIITMadras CyberSecurityCourses StanfordAdvancedCyberSecurityProgram DigitalMarketing&SalesCourses PGPinStrategicDigitalMarketingCourse AdvancedCertificateinStrategicDigitalMarketing AdvancedCertificatePrograminSales SoftwareEngineeringCourses SoftwareDevelopmentIITRoorkee JainOnlineMCA AdvancedSoftwareEngineeringIITMadras DesignThinkingCourses StanfordExecutiveDesignThinking PGinUserExperienceDesignIIT-Bombay FreeCourses DataScienceFreeCourses DataScienceFoundations ProbabilityforDataScience LinearProgrammingforDataScience IntroductiontoR DataVisualizationusingTableau MachineLearningFreeCourses BasicsofMachineLearning StatisticsforMachineLearning FakeNewsDetectionusingMachineLearning ProductCategorizationusingMachineLearning PythonforMachineLearning ArtificialIntelligenceFreeCourses IntroductiontoArtificialIntelligence ArtificialIntelligenceProjects IntroductiontoDeepLearning AIforLeaders IntroductiontoNaturalLanguageProcessing CyberSecurityFreeCourses IntroductiontoCyberSecurity TypesofCyberSecurity CyberSecurityThreats CyberSecurityForLeadership CyberForensics CloudComputingFreeCourses CloudFoundations CloudComputingServiceModels BestCloudCertifications CloudComputingArchitecture ApplicationsofCloudComputing BigDataFreeCourses IntroductiontoBigDataandHadoop MasteringBigDataAnalytics HadoopMapReduce IntroductiontoApacheHive SparkBasics MarketingFreeCourses IntroductiontoDigitalMarketing SEOforBeginners ContentMarketingBasics SearchEngineMarketing AffiliateMarketing EmailMarketing InterviewQuestions PythonInterviewQuestions JavaInterviewQuestions SQLInterviewQuestions MachineLearningInterviewQuestions SeleniumInterviewQuestions NLPInterviewQuestions C++InterviewQuestions AWSInterviewQuestions RPAInterviewQuestions CloudComputingInterviewQuestions Career CareerTopics CareerinAI Careerinpython CareerinCyberSecurity CareerinDigitalMarketer CareerinCloudArchitect CareerinBusinessAnalytics CareerinEthicalHacking Salary DataScientistSalary PythonDeveloperSalary CyberSecuritySalary DataAnalystSalary BusinessAnalystSalary MachineLearningSalary BlockchainDeveloperSalary DigitalMarketingSalary DevOpsSalary Skills DataScientistSkills BigDataAnalyticsSkills CloudArchitectSkills ManagementSkills CloudEngineerSkills AdministrativeSkills DigitalMarketingSkills DevOpsEngineerSkills Resume WebDeveloperResume BusinessAnalystResume DataScienceResume JavaDeveloperResume DigitalMarketingResume PythonDeveloperResume DevOpsEngineerResume ArtificialIntelligenceResume MachineLearningResume Jobs HighestPayingJobsintheWorld HighestPayingFinanceJobs HighestPayingFreelancingJobs HighestPayingITJobs Non–TechnicalJobs OtherResources ArtificialIntelligence MachineLearning PythonProgramming DataScience CloudComputing CyberSecurity BusinessManagement VisualStories More Search AllCourses DataScienceCourses Master’sinDataScience M.TechinDataScience(Bangalore) DataScienceSpecialization PGinBusinessAnalytics DecisionMakingUsingDataScience DataScienceandMachineLearning ArtificialIntelligenceCourses PGP–ArtificialIntelligenceandMachineLearning(Online) PGPinArtificialIntelligenceandMachineLearning(Classroom) PGP–ArtificialIntelligenceforLeaders PGinMachineLearning PGDiplomainComputerScienceandAI–IIITDelhi AIforHighSchoolStudents ManagementCourses ExecutivePGinManagement MBAinDigitalMarketing&DataSciencefromJAIN OnlineMBAinShivNadar BachelorofBusinessfromDeakinUniversity BBADegreeFromJainUniversity ExecutiveMBAfromPES NUSFutureLeadersProgram PGinProductManagementandAnalytics BachelorofBusinessAnalyticsFromDeakin CloudComputingCourses PGPinCloudComputingCourse AdvancedSoftwareEngineeringCourseIITMadras CyberSecurityCourses StanfordAdvancedCyberSecurityProgram DigitalMarketing&SalesCourses PGPinStrategicDigitalMarketingCourse AdvancedCertificateinStrategicDigitalMarketing AdvancedCertificatePrograminSales SoftwareEngineeringCourses SoftwareDevelopmentIITRoorkee JainOnlineMCA AdvancedSoftwareEngineeringIITMadras DesignThinkingCourses StanfordExecutiveDesignThinking PGinUserExperienceDesignIIT-Bombay FreeCourses DataScienceFreeCourses DataScienceFoundations ProbabilityforDataScience LinearProgrammingforDataScience IntroductiontoR DataVisualizationusingTableau MachineLearningFreeCourses BasicsofMachineLearning StatisticsforMachineLearning FakeNewsDetectionusingMachineLearning ProductCategorizationusingMachineLearning PythonforMachineLearning ArtificialIntelligenceFreeCourses IntroductiontoArtificialIntelligence ArtificialIntelligenceProjects IntroductiontoDeepLearning AIforLeaders IntroductiontoNaturalLanguageProcessing CyberSecurityFreeCourses IntroductiontoCyberSecurity TypesofCyberSecurity CyberSecurityThreats CyberSecurityForLeadership CyberForensics CloudComputingFreeCourses CloudFoundations CloudComputingServiceModels BestCloudCertifications CloudComputingArchitecture ApplicationsofCloudComputing BigDataFreeCourses IntroductiontoBigDataandHadoop MasteringBigDataAnalytics HadoopMapReduce IntroductiontoApacheHive SparkBasics MarketingFreeCourses IntroductiontoDigitalMarketing SEOforBeginners ContentMarketingBasics SearchEngineMarketing AffiliateMarketing EmailMarketing InterviewQuestions PythonInterviewQuestions JavaInterviewQuestions SQLInterviewQuestions MachineLearningInterviewQuestions SeleniumInterviewQuestions NLPInterviewQuestions C++InterviewQuestions AWSInterviewQuestions RPAInterviewQuestions CloudComputingInterviewQuestions Career CareerTopics CareerinAI Careerinpython CareerinCyberSecurity CareerinDigitalMarketer CareerinCloudArchitect CareerinBusinessAnalytics CareerinEthicalHacking Salary DataScientistSalary PythonDeveloperSalary CyberSecuritySalary DataAnalystSalary BusinessAnalystSalary MachineLearningSalary BlockchainDeveloperSalary DigitalMarketingSalary DevOpsSalary Skills DataScientistSkills BigDataAnalyticsSkills CloudArchitectSkills ManagementSkills CloudEngineerSkills AdministrativeSkills DigitalMarketingSkills DevOpsEngineerSkills Resume WebDeveloperResume BusinessAnalystResume DataScienceResume JavaDeveloperResume DigitalMarketingResume PythonDeveloperResume DevOpsEngineerResume ArtificialIntelligenceResume MachineLearningResume Jobs HighestPayingJobsintheWorld HighestPayingFinanceJobs HighestPayingFreelancingJobs HighestPayingITJobs Non–TechnicalJobs OtherResources ArtificialIntelligence MachineLearning PythonProgramming DataScience CloudComputing CyberSecurity BusinessManagement VisualStories More Search HomeBusinessAnalyticsTypesofNeuralNetworksandDefinitionofNeuralNetwork Share Facebook Twitter WhatsApp IntroductiontoArtificialNeuralNetworkTypesofNeuralNetworksPerceptronFeedForwardNeuralNetwork MultilayerPerceptronConvolutionalNeuralNetworkRadialBasisFunctionNeuralNetwork RecurrentNeuralNetworkLSTM–LongShort-TermMemorySequencetoSequencemodelsModularNeuralNetworkFAQs Thisblogiscustomtailoredtoaidyourunderstandingondifferenttypesofcommonlyusedneuralnetworks,howtheyworkandtheirindustryapplications.Theblogcommenceswithabriefintroductionontheworkingofneuralnetworks.Wehavetriedtokeepitverysimpleyeteffective. Hereisalistofdifferenttypesofneuralnetworksthatexist: PerceptronFeedForwardNeuralNetworkMultilayerPerceptronConvolutionalNeuralNetworkRadialBasisFunctionalNeuralNetworkRecurrentNeuralNetworkLSTM–LongShort-TermMemorySequencetoSequenceModelsModularNeuralNetwork OurMostPopularFreeCourses: > < Intermediate 2.5Hours IntroductiontoNeuralNetworksandDeepLearning ★ 4.54(657Ratings) Free EnrolNow→ Intermediate 1Hour ConvolutionalNeuralNetworks ★ Good(100+) Free EnrolNow→ Beginner 1Hour IntroductiontoNeuralNetworks ★ Good(100+) Free EnrolNow→ AnIntroductiontoArtificialNeuralNetwork Neuralnetworksrepresentdeeplearningusingartificialintelligence.Certainapplicationscenariosaretooheavyoroutofscopefortraditionalmachinelearningalgorithmstohandle.Astheyarecommonlyknown,NeuralNetworkpitchesinsuchscenariosandfillsthegap.Also,enrolforneuralnetworksanddeeplearningcourse Artificialneuralnetworksareinspiredfromthebiologicalneuronswithinthehumanbodywhichactivateundercertaincircumstancesresultinginarelatedactionperformedbythebodyinresponse.ArtificialneuralnetsconsistofvariouslayersofinterconnectedartificialneuronspoweredbyactivationfunctionswhichhelpinswitchingthemON/OFF.Liketraditionalmachinealgorithms,heretoo,therearecertainvaluesthatneuralnetslearninthetrainingphase. Briefly,eachneuronreceivesamultipliedversionofinputsandrandomweightswhichisthenaddedwithstaticbiasvalue(uniquetoeachneuronlayer),thisisthenpassedtoanappropriateactivationfunctionwhichdecidesthefinalvaluetobegivenoutoftheneuron.Therearevariousactivationfunctionsavailableasperthenatureofinputvalues.Oncetheoutputisgeneratedfromthefinalneuralnetlayer,lossfunction(inputvsoutput)iscalculatedandbackpropagationisperformedwheretheweightsareadjustedtomakethelossminimum.Findingoptimalvaluesofweightsiswhattheoveralloperationisfocusingaround.Pleaserefertothefollowingforbetterunderstanding- Weightsarenumericvalueswhicharemultipliedwithinputs.Inbackpropagation,theyaremodifiedtoreducetheloss.Insimplewords,weightsaremachinelearntvaluesfromNeuralNetworks.Theyself-adjustdependingonthedifferencebetweenpredictedoutputsvstraininginputs.ActivationFunctionisamathematicalformulawhichhelpstheneurontoswitchON/OFF. Inputlayerrepresentsdimensionsoftheinputvector.Hiddenlayerrepresentstheintermediarynodesthatdividetheinputspaceintoregionswith(soft)boundaries.Ittakesinasetofweightedinputandproducesoutputthroughanactivationfunction.Outputlayerrepresentstheoutputoftheneuralnetwork. TypesofNeuralNetworks Therearemanytypesofneuralnetworksavailableorthatmightbeinthedevelopmentstage.Theycanbeclassifieddependingontheir:Structure,Dataflow,Neuronsusedandtheirdensity,Layersandtheirdepthactivationfiltersetc. TypesofNeuralnetwork Wearegoingtodiscussthefollowingneuralnetworks: A.Perceptron Perceptron Perceptronmodel,proposedbyMinsky-PapertisoneofthesimplestandoldestmodelsofNeuron.Itisthesmallestunitofneuralnetworkthatdoescertaincomputationstodetectfeaturesorbusinessintelligenceintheinputdata.Itacceptsweightedinputs,andapplytheactivationfunctiontoobtaintheoutputasthefinalresult.PerceptronisalsoknownasTLU(thresholdlogicunit) Perceptronisasupervisedlearningalgorithmthatclassifiesthedataintotwocategories,thusitisabinaryclassifier.Aperceptronseparatestheinputspaceintotwocategoriesbyahyperplanerepresentedbythefollowingequation AdvantagesofPerceptronPerceptronscanimplementLogicGateslikeAND,OR,orNAND DisadvantagesofPerceptronPerceptronscanonlylearnlinearlyseparableproblemssuchasbooleanANDproblem.Fornon-linearproblemssuchasbooleanXORproblem,itdoesnotwork. B.FeedForwardNeuralNetworks ApplicationsonFeedForwardNeuralNetworks: Simpleclassification(wheretraditionalMachine-learningbasedclassificationalgorithmshavelimitations)Facerecognition[Simplestraightforwardimageprocessing]Computervision[Wheretargetclassesaredifficulttoclassify]SpeechRecognition Thesimplestformofneuralnetworkswhereinputdatatravelsinonedirectiononly,passingthroughartificialneuralnodesandexitingthroughoutputnodes.Wherehiddenlayersmayormaynotbepresent,inputandoutputlayersarepresentthere.Basedonthis,theycanbefurtherclassifiedasasingle-layeredormulti-layeredfeed-forwardneuralnetwork. Numberoflayersdependsonthecomplexityofthefunction.Ithasuni-directionalforwardpropagationbutnobackwardpropagation.Weightsarestatichere.Anactivationfunctionisfedbyinputswhicharemultipliedbyweights.Todoso,classifyingactivationfunctionorstepactivationfunctionisused.Forexample:Theneuronisactivatedifitisabovethreshold(usually0)andtheneuronproduces1asanoutput.Theneuronisnotactivatedifitisbelowthreshold(usually0)whichisconsideredas-1.Theyarefairlysimpletomaintainandareequippedwithtodealwithdatawhichcontainsalotofnoise. OurMostPopularFreeCourses: > < Intermediate 2.5Hours IntroductiontoNeuralNetworksandDeepLearning ★ 4.54(657Ratings) Free EnrolNow→ Intermediate 1Hour ConvolutionalNeuralNetworks ★ Good(100+) Free EnrolNow→ Beginner 1Hour IntroductiontoNeuralNetworks ★ Good(100+) Free EnrolNow→ AdvantagesofFeedForwardNeuralNetworks Lesscomplex,easytodesign&maintainFastandspeedy[One-waypropagation]Highlyresponsivetonoisydata DisadvantagesofFeedForwardNeuralNetworks: Cannotbeusedfordeeplearning[duetoabsenceof denselayersandbackpropagation] C.MultilayerPerceptron  ApplicationsonMulti-LayerPerceptron SpeechRecognitionMachineTranslationComplexClassification Anentrypointtowardscomplexneuralnetswhereinputdatatravelsthroughvariouslayersofartificialneurons. Everysinglenodeisconnectedtoallneuronsinthenextlayerwhichmakesitafullyconnectedneuralnetwork.InputandoutputlayersarepresenthavingmultiplehiddenLayersi.e.atleastthreeormorelayersintotal.Ithasabi-directionalpropagationi.e.forwardpropagationandbackwardpropagation. Inputsaremultipliedwithweightsandfedtotheactivationfunctionandinbackpropagation,theyaremodifiedtoreducetheloss.Insimplewords,weightsaremachinelearntvaluesfromNeuralNetworks.Theyself-adjustdependingonthedifferencebetweenpredictedoutputsvstraininginputs.Nonlinearactivationfunctionsareusedfollowedbysoftmaxasanoutputlayeractivationfunction. AdvantagesonMulti-LayerPerceptron Usedfordeeplearning[duetothepresenceofdensefully connectedlayersandbackpropagation]  DisadvantagesonMulti-LayerPerceptron:  Comparativelycomplextodesignandmaintain Comparativelyslow(dependsonnumberof hiddenlayers) D.ConvolutionalNeuralNetwork ApplicationsonConvolutionNeuralNetwork ImageprocessingComputerVisionSpeechRecognitionMachinetranslation Convolutionneuralnetworkcontainsathree-dimensionalarrangementofneurons,insteadofthestandardtwo-dimensionalarray.Thefirstlayeriscalledaconvolutionallayer.Eachneuronintheconvolutionallayeronlyprocessestheinformationfromasmallpartofthevisualfield.Inputfeaturesaretakeninbatch-wiselikeafilter.Thenetworkunderstandstheimagesinpartsandcancomputetheseoperationsmultipletimestocompletethefullimageprocessing.ProcessinginvolvesconversionoftheimagefromRGBorHSIscaletogrey-scale.Furtheringthechangesinthepixelvaluewillhelptodetecttheedgesandimagescanbeclassifiedintodifferentcategories. Propagationisuni-directionalwhereCNNcontainsoneormoreconvolutionallayersfollowedbypoolingandbidirectionalwheretheoutputofconvolutionlayergoestoafullyconnectedneuralnetworkforclassifyingtheimagesasshownintheabovediagram.Filtersareusedtoextractcertainpartsoftheimage.InMLPtheinputsaremultipliedwithweightsandfedtotheactivationfunction.ConvolutionusesRELUandMLPusesnonlinearactivationfunctionfollowedbysoftmax.Convolutionneuralnetworksshowveryeffectiveresultsinimageandvideorecognition,semanticparsingandparaphrasedetection. AdvantagesofConvolutionNeuralNetwork: UsedfordeeplearningwithfewparametersLessparameterstolearnascomparedtofully connectedlayer DisadvantagesofConvolutionNeuralNetwork: ComparativelycomplextodesignandmaintainComparativelyslow[dependsonthenumberofhiddenlayers] E.RadialBasisFunctionNeuralNetworks  RadialBasisFunctionNetworkconsistsofaninputvectorfollowedbyalayerofRBFneuronsandanoutputlayerwithonenodepercategory.Classificationisperformedbymeasuringtheinput’ssimilaritytodatapointsfromthetrainingsetwhereeachneuronstoresaprototype.Thiswillbeoneoftheexamplesfromthetrainingset. Whenanewinputvector[then-dimensionalvectorthatyouaretryingtoclassify]needstobeclassified,eachneuroncalculatestheEuclideandistancebetweentheinputanditsprototype.Forexample,ifwehavetwoclassesi.e.classAandClassB,thenthenewinputtobeclassifiedismoreclosetoclassAprototypesthantheclassBprototypes.Hence,itcouldbetaggedorclassifiedasclassA. EachRBFneuroncomparestheinputvectortoitsprototypeandoutputsavaluerangingwhichisameasureofsimilarityfrom0to1.Astheinputequalstotheprototype,theoutputofthatRBFneuronwillbe1andwiththedistancegrowsbetweentheinputandprototypetheresponsefallsoffexponentiallytowards0.Thecurvegeneratedoutofneuron’sresponsetendstowardsatypicalbellcurve.Theoutputlayerconsistsofasetofneurons[onepercategory]. Application:PowerRestorationa.PowercutP1needstoberestoredfirstb.PowercutP3needstoberestorednext,asitimpactsmorehousesc.PowercutP2shouldbefixedlastasitimpactsonlyonehouse F.RecurrentNeuralNetworks ApplicationsofRecurrentNeuralNetworks Textprocessinglikeautosuggest,grammarchecks,etc.TexttospeechprocessingImagetaggerSentimentAnalysisTranslationDesignedtosavetheoutputofalayer,RecurrentNeuralNetworkisfedbacktotheinputtohelpinpredictingtheoutcomeofthelayer.Thefirstlayeristypicallyafeedforwardneuralnetworkfollowedbyrecurrentneuralnetworklayerwheresomeinformationithadintheprevioustime-stepisrememberedbyamemoryfunction.Forwardpropagationisimplementedinthiscase.Itstoresinformationrequiredforit’sfutureuse.Ifthepredictioniswrong,thelearningrateisemployedtomakesmallchanges.Hence,makingitgraduallyincreasetowardsmakingtherightpredictionduringthebackpropagation. AdvantagesofRecurrentNeuralNetworks Modelsequentialdatawhere eachsamplecanbeassumedtobedependentonhistoricalonesisoneoftheadvantage.  Usedwithconvolution layerstoextendthepixeleffectiveness. DisadvantagesofRecurrentNeuralNetworks Gradientvanishingandexplodingproblems Trainingrecurrentneuralnetscouldbeadifficult task DifficulttoprocesslongsequentialdatausingReLUasanactivationfunction. ImprovementoverRNN:LSTM(LongShort-TermMemory)Networks LSTMnetworksareatypeofRNNthatusesspecialunitsinadditiontostandardunits.LSTMunitsincludea‘memorycell’thatcanmaintaininformationinmemoryforlongperiodsoftime.Asetofgatesisusedtocontrolwheninformationentersthememorywhenit’soutput,andwhenit’sforgotten.Therearethreetypesofgatesviz,Inputgate,outputgateandforgetgate.Inputgatedecideshowmanyinformationfromthelastsamplewillbekeptinmemory;theoutputgateregulatestheamountofdatapassedtothenextlayer,andforgetgatescontrolthetearingrateofmemorystored.Thisarchitectureletsthemlearnlonger-termdependencies ThisisoneoftheimplementationsofLSTMcells,manyotherarchitecturesexist. source:https://www.researchgate.net/figure/RNN-simple-cell-versus-LSTM-cell-4_fig2_317954962 G.Sequencetosequencemodels   AsequencetosequencemodelconsistsoftwoRecurrentNeuralNetworks.Here,thereexistsanencoderthatprocessestheinputandadecoderthatprocessestheoutput.Theencoderanddecoderworksimultaneously–eitherusingthesameparameterordifferentones.Thismodel,oncontrarytotheactualRNN,isparticularlyapplicableinthosecaseswherethelengthoftheinputdataisequaltothelengthoftheoutputdata.WhiletheypossesssimilarbenefitsandlimitationsoftheRNN,thesemodelsareusuallyappliedmainlyinchatbots,machinetranslations,andquestionansweringsystems. OurMostPopularFreeCourses: > < Intermediate 2.5Hours IntroductiontoNeuralNetworksandDeepLearning ★ 4.54(657Ratings) Free EnrolNow→ Intermediate 1Hour ConvolutionalNeuralNetworks ★ Good(100+) Free EnrolNow→ Beginner 1Hour IntroductiontoNeuralNetworks ★ Good(100+) Free EnrolNow→ H.ModularNeuralNetwork  ApplicationsofModularNeuralNetwork StockmarketpredictionsystemsAdaptiveMNNforcharacterrecognitions Compressionofhighlevelinputdata Amodularneuralnetworkhasanumberofdifferentnetworksthatfunctionindependentlyandperformsub-tasks.Thedifferentnetworksdonotreallyinteractwithorsignaleachotherduringthecomputationprocess.Theyworkindependentlytowardsachievingtheoutput. Asaresult,alargeandcomplexcomputationalprocessaredonesignificantlyfasterbybreakingitdownintoindependentcomponents.Thecomputationspeedincreasesbecausethenetworksarenotinteractingwithorevenconnectedtoeachother. AdvantagesofModularNeuralNetwork EfficientIndependenttrainingRobustness DisadvantagesofModularNeuralNetwork MovingtargetProblems FAQs 1.Whatare3majorcategoriesofneuralnetworks? Thethreemostimportanttypesofneuralnetworksare:ArtificialNeuralNetworks(ANN);ConvolutionNeuralNetworks(CNN),andRecurrentNeuralNetworks(RNN). 2.Whatisneuralnetworkanditstypes? NeuralNetworks are artificialnetworks usedinMachineLearningthatworkinasimilarfashiontothehumannervoussystem.Manythingsareconnectedinvariouswaysforaneuralnetworktomimicandworklikethehumanbrain.Neuralnetworksarebasicallyusedincomputationalmodels. 3.WhatisCNNandDNN? Adeep neuralnetwork (DNN)isanartificial neuralnetwork (ANN)withmultiplelayersbetweentheinputandoutputlayers.Theycanmodelcomplexnon-linearrelationships.ConvolutionalNeuralNetworks(CNN)areanalternativetypeof DNN thatallowmodellingbothtimeandspacecorrelationsinmultivariatesignals. 4.HowdoesCNNdifferfromAnn? CNNisaspecifickindofANNthathasoneormorelayersofconvolutionalunits.Theclassof ANN coversseveralarchitecturesincludingConvolutionalNeuralNetworks(CNN),RecurrentNeuralNetworks(RNN)egLSTMandGRU,Autoencoders,andDeepBeliefNetworks. 5.WhyisCNNbetterthanMLP? MultilayerPerceptron(MLP) isgreatforMNISTasitisasimplerandmorestraightforwarddataset,butitlags whenitcomestoreal-worldapplicationincomputervision,specificallyimageclassificationascomparedtoCNNwhichisgreat. Hopeyoufoundthisinteresting!YoucancheckoutourblogaboutConvolutionalNeuralNetwork.Tolearnmoreaboutsuchconcepts,takeupanartificialintelligenceonlinecourseandupskilltoday. 34 RELATEDARTICLESMOREFROMAUTHOR BusinessAnalytics UpskillwithBusinessAnalyticsCourses  BusinessAnalytics BestBusinessAnalyticsCourse:TopReasonswhyyoushouldenrollinBachelorofBusinessAnalytics BusinessAnalytics BusinessAnalyst:JobDescription,Skills,responsibilities,andsalarytrends LEAVEAREPLYCancelreply Pleaseenteryourcomment! Pleaseenteryournamehere Youhaveenteredanincorrectemailaddress! Pleaseenteryouremailaddresshere Savemyname,email,andwebsiteinthisbrowserforthenexttimeIcomment. Pleaseenterananswerindigits:six−5= Δ GreatLearningisaned-techcompanythatoffersimpactfulandindustry-relevantprogramsinhigh-growthareas.Withastrongpresenceacrosstheglobe,wehaveempowered10,000+learnersfromover50countriesinachievingpositiveoutcomesfortheircareers.KnowMore FacebookInstagramLinkedinPinterestTwitter ©2021GreatLearningAllrightsreserved34