Real-Life Applications of Neural Networks | Smartsheet

Neural networks are fundamental to deep learning, a robust set of NN techniques that lends itself to solving abstract problems, such as bioinformatics, drug ... Skiptomaincontent Real-LifeandBusinessApplicationsofNeuralNetworks SmartsheetContributor DianaRamos Oct17,2018 (updatedJul19,2021) TrySmartsheetforFree GetaFreeSmartsheetDemo Today,neuralnetworks(NN)arerevolutionizingbusinessandeverydaylife,bringingustothenextlevelinartificialintelligence(AI).Byemulatingthewayinterconnectedbraincellsfunction,NN-enabledmachines(includingthesmartphonesandcomputersthatweuseonadailybasis)arenowtrainedtolearn,recognizepatterns,andmakepredictionsinahumanoidfashionaswellassolveproblemsineverybusinesssector. Inthisarticle,weofferthemostusefulguidetoneuralnetworks’essentialalgorithms,dependenceonbigdata,latestinnovations,andfuture.Weincludeinsideinformationfrompioneers,applicationsforengineeringandbusiness,andadditionalresources. InThisArticle WhatAreNeuralNetworks? HowtheBiologicalModelofNeuralNetworksFunctions HowArtificialNeuralNetworksFunction HowDoYouTrainaNeuralNetwork? ABriefHistoryofNeuralNetworks WhyDoWeUseNeuralNetworks? Real-WorldandIndustryApplicationsofNeuralNetworks WhatAretheTypesofNeuralNetworks? WhatAreNeuralNetworksinDataMining? Neuralvs.ConventionalComputers TheChallengesofNeuralNetworks TheFutureofNeuralNetworks ResourcesonNeuralNetworks TheFutureofWorkwithAutomatedProcessesinSmartsheet WhatAreNeuralNetworks? Abranchofmachinelearning,neuralnetworks(NN),alsoknownasartificialneuralnetworks(ANN),arecomputationalmodels—essentiallyalgorithms.Neuralnetworkshaveauniqueabilitytoextractmeaningfromimpreciseorcomplexdatatofindpatternsanddetecttrendsthataretooconvolutedforthehumanbrainorforothercomputertechniques.Neuralnetworkshaveprovideduswithgreaterconvenienceinnumerousways,includingthroughridesharingapps,Gmailsmartsorting,andsuggestionsonAmazon. Themostgroundbreakingaspectofneuralnetworksisthatoncetrained,theylearnontheirown.Inthisway,theyemulatehumanbrains,whicharemadeupofneurons,thefundamentalbuildingblockofbothhumanandneuralnetworkinformationtransmission.   “Humanbrainsandartificialneuralnetworksdolearnsimilarly,”explainsAlexCardinell,FounderandCEOofCortx,anartificialintelligencecompanythatusesneuralnetworksinthedesignofitsnaturallanguageprocessingsolutions,includinganautomatedgrammarcorrectionapplication,PerfectTense.“Inbothcases,neuronscontinuallyadjusthowtheyreactbasedonstimuli.Ifsomethingisdonecorrectly,you’llgetpositivefeedbackfromneurons,whichwillthenbecomeevenmorelikelytotriggerinasimilar,futureinstance.Conversely,ifneuronsreceivenegativefeedback,eachofthemwilllearntobelesslikelytotriggerinafutureinstance,”henotes.   SeehowSmartsheetcanhelpyoubemoreeffective Watchthedemotoseehowyoucanmoreeffectivelymanageyourteam,projects,andprocesseswithreal-timeworkmanagementinSmartsheet.   Watchafreedemo HowtheBiologicalModelofNeuralNetworksFunctions Whatareneuralnetworksemulatinginhumanbrainstructure,andhowdoestrainingwork? Allmammalianbrainsconsistofinterconnectedneuronsthattransmitelectrochemicalsignals.Neuronshaveseveralcomponents:thebody,whichincludesanucleusanddendrites;axons,whichconnecttoothercells;andaxonterminalsorsynapses,whichtransmitinformationorstimulifromoneneurontoanother.Combined,thisunitcarriesoutcommunicationandintegrationfunctionsinthenervoussystem.Thehumanbrainhasamassivenumberofprocessingunits(86billionneurons)thatenabletheperformanceofhighlycomplexfunctions.   HowArtificialNeuralNetworksFunction ANNsarestatisticalmodelsdesignedtoadaptandself-programbyusinglearningalgorithmsinordertounderstandandsortoutconcepts,images,andphotographs.Forprocessorstodotheirwork,developersarrangetheminlayersthatoperateinparallel.Theinputlayerisanalogoustothedendritesinthehumanbrain’sneuralnetwork.Thehiddenlayeriscomparabletothecellbodyandsitsbetweentheinputlayerandoutputlayer(whichisakintothesynapticoutputsinthebrain).Thehiddenlayeriswhereartificialneuronstakeinasetofinputsbasedonsynapticweight,whichistheamplitudeorstrengthofaconnectionbetweennodes.Theseweightedinputsgenerateanoutputthroughatransferfunctiontotheoutputlayer.   HowDoYouTrainaNeuralNetwork? Onceyou’vestructuredanetworkforaparticularapplication,training(i.e.,learning),begins.Therearetwoapproachestotraining.Supervisedlearningprovidesthenetworkwithdesiredoutputsthroughmanualgradingofnetworkperformanceorbydeliveringdesiredoutputsandinputs.Unsupervisedlearningoccurswhenthenetworkmakessenseofinputswithoutoutsideassistanceorinstruction. There’sstillalongwaytogointheareaofunsupervisedlearning.“Gettinginformationfromunlabeleddata,[aprocess]wecallunsupervisedlearning,isaveryhottopicrightnow,butclearlynotsomethingwehavecrackedyet.It’ssomethingthatstillfallsinthechallengecolumn,”observesUniversitédeMontréal’sYoshuaBengiointhearticle“TheRiseofNeuralNetworksandDeepLearninginOurEverydayLives.” Bengioisreferringtothefactthatthenumberofneuralnetworkscan’tmatchthenumberofconnectionsinthehumanbrain,buttheformer’sabilitytocatchupmaybejustoverthehorizon.Moore’sLaw,whichstatesthatoverallprocessingpowerforcomputerswilldoubleeverytwoyears,givesusahintaboutthedirectioninwhichneuralnetworksandAIareheaded.IntelCEOBrianKrzanichaffirmedatthe2017ComputerElectronicsShowthat“Moore’sLawisaliveandwellandflourishing.”Sinceitsinceptioninthemid-20thcentury,neuralnetworks’abilityto“think”hasbeenchangingourworldatanincrediblepace. ABriefHistoryofNeuralNetworks Neuralnetworksdatebacktotheearly1940swhenmathematiciansWarrenMcCullochandWalterPittsbuiltasimplealgorithm-basedsystemdesignedtoemulatehumanbrainfunction.Workinthefieldacceleratedin1957whenCornellUniversity’sFrankRosenblattconceivedoftheperceptron,thegroundbreakingalgorithmdevelopedtoperformcomplexrecognitiontasks.Duringthefourdecadesthatfollowed,thelackofcomputingpowernecessarytoprocesslargeamountsofdataputthebrakesonadvances.Inthe2000s,thankstotheadventofgreatercomputingpowerandmoresophisticatedhardware,aswellastotheexistenceofvastdatasetstodrawfrom,computerscientistsfinallyhadwhattheyneeded,andneuralnetworksandAItookoff,withnoendinsight.Tounderstandhowmuchthefieldhasexpandedinthenewmillennium,considerthatninetypercentofinternetdatahasbeencreatedsince2016.Thatpacewillcontinuetoaccelerate,thankstothegrowthoftheInternetofThings(IoT). Formorebackgroundandanexpansivetimeline,read“TheDefinitiveGuidetoMachineLearning:BusinessApplications,Techniques,andExamples.” WhyDoWeUseNeuralNetworks? Neuralnetworks’human-likeattributesandabilitytocompletetasksininfinitepermutationsandcombinationsmakethemuniquelysuitedtotoday’sbigdata-basedapplications.Becauseneuralnetworksalsohavetheuniquecapacity(knownasfuzzylogic)tomakesenseofambiguous,contradictory,orincompletedata,theyareabletousecontrolledprocesseswhennoexactmodelsareavailable. AccordingtoareportpublishedbyStatista,in2017,globaldatavolumesreachedcloseto100,000petabytes(i.e.,onemilliongigabytes)permonth;theyareforecastedtoreach232,655petabytesby2021.Withbusinesses,individuals,anddevicesgeneratingvastamountsofinformation,allofthatbigdataisvaluable,andneuralnetworkscanmakesenseofit. AttributesofNeuralNetworks Withthehuman-likeabilitytoproblem-solve—andapplythatskilltohugedatasets—neuralnetworkspossessthefollowingpowerfulattributes: AdaptiveLearning:Likehumans,neuralnetworksmodelnon-linearandcomplexrelationshipsandbuildonpreviousknowledge.Forexample,softwareusesadaptivelearningtoteachmathandlanguagearts. Self-Organization:Theabilitytoclusterandclassifyvastamountsofdatamakesneuralnetworksuniquelysuitedfororganizingthecomplicatedvisualproblemsposedbymedicalimageanalysis. Real-TimeOperation:Neuralnetworkscan(sometimes)providereal-timeanswers,asisthecasewithself-drivingcarsanddronenavigation. Prognosis:NN’sabilitytopredictbasedonmodelshasawiderangeofapplications,includingforweatherandtraffic. FaultTolerance:Whensignificantpartsofanetworkarelostormissing,neuralnetworkscanfillintheblanks.Thisabilityisespeciallyusefulinspaceexploration,wherethefailureofelectronicdevicesisalwaysapossibility. TasksNeuralNetworksPerform Neuralnetworksarehighlyvaluablebecausetheycancarryouttaskstomakesenseofdatawhileretainingalltheirotherattributes.Herearethecriticaltasksthatneuralnetworksperform: Classification:NNsorganizepatternsordatasetsintopredefinedclasses. Prediction:Theyproducetheexpectedoutputfromgiveninput. Clustering:Theyidentifyauniquefeatureofthedataandclassifyitwithoutany knowledgeofpriordata. Associating:Youcantrainneuralnetworksto"remember"patterns.Whenyoushowanunfamiliarversionofapattern,thenetworkassociatesitwiththemostcomparableversioninitsmemoryandrevertstothelatter. Neuralnetworksarefundamentaltodeeplearning,arobustsetofNNtechniquesthatlendsitselftosolvingabstractproblems,suchasbioinformatics,drugdesign,socialnetworkfiltering,andnaturallanguagetranslation.Deeplearningiswherewewillsolvethemostcomplicatedissuesinscienceandengineering,includingadvancedrobotics.Asneuralnetworksbecomesmarterandfaster,wemakeadvancesonadailybasis. Real-WorldandIndustryApplicationsofNeuralNetworks AsanAugust2018NewYorkTimesarticlenotes,“Thecompaniesandgovernmentagenciesthathavebegunenlistingtheautomationsoftwarerunthegamut.TheyincludeGeneralMotors,BMW,GeneralElectric,Unilever,MasterCard,Manpower,FedEx,Cisco,Google,theDefenseDepartment,andNASA.”We’rejustseeingthebeginningofneuralnetwork/AIapplicationschangingthewayourworldworks. H3:EngineeringApplicationsofNeuralNetworks Engineeringiswhereneuralnetworkapplicationsareessential,particularlyinthe“highassurancesystemsthathaveemergedinvariousfields,includingflightcontrol,chemicalengineering,powerplants,automotivecontrol,medicalsystems,andothersystemsthatrequireautonomy.”(Source:ApplicationofNeuralNetworksinHighAssuranceSystems:ASurvey.) Weaskedtwoexpertsintheengineeringsectorabouthowtheirapplicationsimproveretail,manufacturing,oilandgas,navigation,andinformationretrievalinofficeenvironments.   Peopleusewirelesstechnology,whichallowsdevicestoconnecttotheinternetorcommunicatewithoneanotherwithinaparticulararea,inmanydifferentfieldstoreducecostsandenhanceefficiency.HuwRees,VPofSales&MarketingforKodaCloud,anapplicationdesignedtooptimizeWi-Fiperformance,describesjustsomeuses. ReesofferssomeeverydayexamplesofWi-Fiuse:“SupermarketchainsuseWi-Fiscannerstoscanproduceinandoutoftheirdistributioncentersandindividualmarkets.IftheWi-Fiisn’tworkingwell,entirebusinessesaredisrupted.ManufacturingandoilandgasconcernsarealsogoodexamplesofbusinesseswhereWi-Fiismissioncritical,becauseensuringreliabilityandoptimizationisanabsoluterequirement,”hesays. Wi-Fiisgreat,butittakesalotofoversighttodoitsjob.“Mostenterpriseorlarge-scalewirelesslocalareanetworksolutionsrequirenear-constantmonitoringandadjustmentbyhighlytrainedWi-Fiexperts,anexpensivewaytoensurethenetworkisperformingoptimally,”Reespointsout.“KodaCloudsolvesthatproblemthroughanintelligentsystemthatusesalgorithmsandthroughadaptivelearning,whichgeneratesaself-improvingloop,”headds. ReesshareshowKodaCloudtechnologytakesadvantageofneuralnetworkstocontinuouslyimprove:“Thenetworklearnsandself-healsbasedonbothglobalandlocallearning.Here’saglobalexample:ThesystemlearnsthatanewAndroidoperatingsystemhasbeendeployedandrequiresadditionalconfigurationandthresholdchangestoworkoptimally.Oncethesystemhasmadeadjustmentsandmeasuringimprovementsnecessitatedbythisupgrade,itappliesthisknowledgetoallotherKodaCloudcustomersinstantaneously,sothesystemimmediatelyrecognizesanysimilardeviceandsolvesissues.Foralocalexample,let’ssaythesystemlearnsthelocalradiofrequencyenvironmentforeachaccesspoint.Eachdevicethenconnectstoeachaccesspoint,whichresultsinthresholdchangestolocaldeviceradioparameters.Globallyandlocally,theprocessisacontinuouscycletooptimizeWi-Fiqualityforeverydevice.”   Afast-developingtechnology,dronesareusedindisasterrelief,oil,gas,andmineralexploration,aerialsurveillance,realestateandconstruction,andfilmmaking.NeillMcOran-CampbellisCEOofAeiou.tech,whichdesignsadvanceddronetechnologyforuseinmanydifferentsectors.“OurDawnplatformisanon-boardseriesofsensorsandacompanioncomputerthatallowsvirtuallyanyunmannedaerialsystemtoutilizethewiderangeofbenefitsthatAIoffers,fromflightmechanics,suchasnavigationandobstacleavoidance,toservicessuchasinfrastructureinspectionorpackagedelivery,”saysMcOran-Campbell. McOran-CampbellexplainshowDawnfunctionsbasedontwolevelsofbiology:“Atthefirstlevel,weuseANNstoprocessrawinformation.Therearethreedifferenttypesofnetworksweuse:recurrentneuralnetworks,whichusethepasttoinformpredictionsaboutthefuture;convolutionalneuralnetworks,whichuse‘sliding’bundlesofneurons(wegenerallyusethistypetoprocessimagery);andmoreconventionalneuralnetworks,i.e.,actualnetworksofneurons.Conventionalneuralnetworksareveryusefulforproblemslikenavigation,especiallywhentheyarecombinedwithrecurrentelements. “Atthemoresophisticated,secondlevel,Dawn’sstructureemulatesthebestarchitecturethatexistsforprocessinginformation:thehumanbrain.Thisallowsustobreakdownthehighlycomplexproblemofautonomythesamewaybiologydoes:withcompartmentalized‘cortexes,’eachonewiththeirneuralnetworksandeachwiththeircommunicationpathwaysandhierarchicalcommandstructures.Theresultisthatinformationflowsinwavesthroughthecortexesinthesamewaythatitdoesinthebrain.[Inbothinstances,theprocessisoptimized]foreffectivenessandefficiencyininformationprocessing,”heexplains. Here’salistofotherneuralnetworkengineeringapplicationscurrentlyinuseinvariousindustries: Aerospace:Aircraftcomponentfaultdetectorsandsimulations,aircraftcontrolsystems,high-performanceauto-piloting,andflightpathsimulations Automotive:Improvedguidancesystems,developmentofpowertrains,virtualsensors,andwarrantyactivityanalyzers Electronics:Chipfailureanalysis,circuitchiplayouts,machinevision,non-linearmodeling,predictionofthecodesequence,processcontrol,andvoicesynthesis Manufacturing:Chemicalproductdesignanalysis,dynamicmodelingofchemicalprocesssystems,processcontrol,processandmachinediagnosis,productdesignandanalysis,paperqualityprediction,projectbidding,planningandmanagement,qualityanalysisofcomputerchips,visualqualityinspectionsystems,andweldingqualityanalysis Mechanics:Conditionmonitoring,systemsmodeling,andcontrol Robotics:Forkliftrobots,manipulatorcontrollers,trajectorycontrol,andvisionsystems Telecommunications:ATMnetworkcontrol,automatedinformationservices,customerpaymentprocessingsystems,datacompression,equalizers,faultmanagement,handwritingrecognition,networkdesign,management,routingandcontrol,networkmonitoring,real-timetranslationofspokenlanguage,andpatternrecognition(faces,objects,fingerprints,semanticparsing,spellcheck,signalprocessing,andspeechrecognition) BusinessApplicationsofNeuralNetworks: Real-worldbusinessapplicationsforneuralnetworksarebooming.Insomecases,NNshavealreadybecomethemethodofchoiceforbusinessesthatusehedgefundanalytics,marketingsegmentation,andfrauddetection.Herearesomeneuralnetworkinnovatorswhoarechangingthebusinesslandscape.   Atatimewhenfindingqualifiedworkersforparticularjobsisbecomingincreasinglydifficult,especiallyinthetechsector,neuralnetworksandAIaremovingtheneedle.EdDonner,Co-FounderandCEOofuntapt,usesneuralnetworksandAItosolvetalentandhumanresourceschallenges,suchashiringinefficiency,pooremployeeretention,dissatisfactionwithwork,andmore.“Intheend,wecreatedadeeplearningmodelthatcanmatchpeopletoroleswherethey’remorelikelytosucceed,allinamatterofmilliseconds,”Donnerexplains. “NeuralnetsandAIhaveincrediblescope,andyoucanusethemtoaidhumandecisionsinanysector.Deeplearningwasn’tthefirstsolutionwetested,butit’sconsistentlyoutperformedtherestinpredictingandimprovinghiringdecisions.Wetrainedour16-layerneuralnetworkonmillionsofdatapointsandhiringdecisions,soitkeepsgettingbetterandbetter.That’swhyI’manadvocateforeverycompanytoinvestinAIanddeeplearning,whetherinHRoranyothersector.Businessisbecomingmoreandmoredatadriven,socompanieswillneedtoleverageAItostaycompetitive,”Donnerrecommends. Thefieldofneuralnetworksanditsuseofbigdatamaybehigh-tech,butitsultimatepurposeistoservepeople.Insomeinstances,thelinktohumanbenefitsisverydirect,asisthecasewithOKRA’sartificialintelligenceservice.   “OKRA’splatformhelpshealthcarestakeholdersandbiopharmamakebetter,evidence-baseddecisionsinreal-time,anditanswersbothtreatment-relatedandbrandquestionsfordifferentmarkets,”emphasizesLoubnaBouarfa,CEOandFounderofOkraTechnologiesandanappointeetotheEuropeanCommission'sHigh-LevelExpertGrouponAI.“Infostercare,weapplyneuralnetworksandAItomatchchildrenwithfostercaregiverswhowillprovidemaximumstability.Wealsoapplythetechnologiestoofferreal-timedecisionsupporttosocialcaregiversandthefosterfamilyinordertobenefitchildren,”shecontinues. LikemanyAIcompanies,OKRAleveragesitstechnologytomakepredictionsusingmultiple,bigdatasources,includingCRM,medicalrecords,andconsumer,sales,andbrandmeasurements.Then,Bouarfaexplains,“Weusestate-of-the-artmachinelearningalgorithms,suchasdeepneuralnetworks,ensemblelearning,topicrecognition,andawiderangeofnon-parametricmodelsforpredictiveinsightsthatimprovehumanlives.” AccordingtotheWorldCancerResearchFund,melanomaisthe19thmostcommoncancerworldwide.Oneinfivepeopleontheplanetdevelopskincancer,andearlydetectionisessentialtopreventskincancer-relateddeath.There’sanappforthat:aphoneapptoperformphotoself-checksusingasmartphone.   “SkinVisionusesourproprietarymathematicalalgorithmtobuildastructuralmapthatrevealsthedifferentgrowthpatternsofthetissuesinvolved,”saysMatthewEnevoldson,SkinVision’sPublicRelationsManager. Enevoldsonaddsthatthephoneappworksfast:“Injust30seconds,theappindicateswhichspotsontheskinneedtobetrackedovertimeandgivestheimagealow,medium,orhigh-riskindication.Themostrecentdatashowsthatourservicehasaspecificityof80percentandasensitivityof94percent,wellabovethatofadermatologist(asensitivityof75percent),aspecialistdermatologist(asensitivityof92percent),orageneralpractitioner(asensitivityof60percent).Everyphotoisdouble-checkedbyourteamofimagerecognitionexpertsanddermatologistsforqualitypurposes.High-riskphotosareflagged,and,within48hours,usersreceivepersonalmedicaladvicefromadoctoraboutnextsteps.”Theapphas1.2millionusersworldwide.   Keepingtrackofdatainanyworkenvironmentandmakinggooduseofitcanbeachallenge.RobMayisCEOandCo-FounderofTalla,acompanythatbuilds“digitalworkers”thatassistemployeeswithdailytasksaroundinformationretrieval,access,andupkeep.“WegivebusinessestheabilitytoadoptAIinameaningfulwayandstartrealizingimmediateimprovementstoemployeeproductivityandknowledgesharingacrosstheorganization,”Mayexplains.“IfacompanystorestheirproductdocumentationinTalla,itssalesrepscaninstantlyaccessthatinformationwhileonsalescalls.Thisabilitytoimmediatelyandeasilyaccessaccurate,verified,up-to-dateinformationhasadirectimpactonrevenue.Byhavinginformationdeliveredtoemployeeswhentheyneedit,theprocessofonboardingandtrainingnewrepsbecomesbetter,faster,andlessexpensive.” Talla’sneuralnetworktechnologydrawsondifferentlearningapproaches.“Weusesemanticmatching,neuralmachinetranslation,activelearning,andtopicmodelingtolearnwhat’srelevantandimportanttoyourorganization,andwedeliverabetterexperienceovertime,”hesays.MaydifferentiatesTalla’stakeonAI:“ThistechnologyhasliftedthehoodonAI,allowinguserstotrainknowledge-basedcontentwithadvancedAItechniques.Tallagivesusersthepowertomaketheirinformationmorediscoverable,actionable,andrelevanttoemployees.ContentcreatorscantrainTallatoidentifysimilarcontent,answerquestions,andidentifyknowledgegaps.” HerearefurthercurrentexamplesofNNbusinessapplications: Banking:Creditcardattrition,creditandloanapplicationevaluation,fraudandriskevaluation,andloandelinquencies BusinessAnalytics:Customerbehaviormodeling,customersegmentation,fraudpropensity,marketresearch,marketmix,marketstructure,andmodelsforattrition,default,purchase,andrenewals Defense:Counterterrorism,facialrecognition,featureextraction,noisesuppression,objectdiscrimination,sensors,sonar,radarandimagesignalprocessing,signal/imageidentification,targettracking,andweaponsteering Education:Adaptivelearningsoftware,dynamicforecasting,educationsystemanalysisandforecasting,studentperformancemodeling,andpersonalityprofiling Financial:Corporatebondratings,corporatefinancialanalysis,creditlineuseanalysis,currencypriceprediction,loanadvising,mortgagescreening,realestateappraisal,andportfoliotrading Medical:Cancercellanalysis,ECGandEEGanalysis,emergencyroomtestadvisement,expensereductionandqualityimprovementforhospitalsystems,transplantprocessoptimization,andprosthesisdesign Securities:Automaticbondrating,marketanalysis,andstocktradingadvisorysystems Transportation:Routingsystems,truckbrakediagnosissystems,andvehiclescheduling Theuseofneuralnetworksseemsunstoppable.“Withtheadvancementofcomputerandcommunicationtechnologies,thewholeprocessofdoingbusinesshasundergoneamassivechange.Moreandmoreknowledge-basedsystemshavemadetheirwayintoalargenumberofcompanies,”researchersNikhilBhargavaandManikGuptafoundin"ApplicationofArtificialNeuralNetworksinBusinessApplications." WhatAretheTypesofNeuralNetworks? Neuralnetworksaresetsofalgorithmsintendedtorecognizepatternsandinterpretdatathroughclusteringorlabeling.Inotherwords,neuralnetworksarealgorithms.Atrainingalgorithmisthemethodyouusetoexecutetheneuralnetwork’slearningprocess.Asthereareahugenumberoftrainingalgorithmsavailable,eachconsistingofvariedcharacteristicsandperformancecapabilities,youusedifferentalgorithmstoaccomplishdifferentgoals.  Collectively,machinelearningengineersdevelopmanythousandsofnewalgorithmsonadailybasis.Usually,thesenewalgorithmsarevariationsonexistingarchitectures,andtheyprimarilyusetrainingdatatomakeprojectionsorbuildreal-worldmodels. Here’saguidetosomeoftoday’scommonneuralnetworkalgorithms.Forgreaterclarityaroundunfamiliarterms,youcanrefertotheglossariesintheresourcesectionofthisarticle. ALayman’sGuidetoCommonNeuralNetworkAlgorithms Algorithm Purpose   Autoencoder(AE) YoutypicallyuseAEstoreducethenumberofrandomvariablesunderconsideration,sothesystemcanlearnarepresentationforasetofdataand,therefore,processgenerativedatamodels. BidirectionalRecurrentNeuralNetwork(BRNN) ThegoalofaBRNNistoincreasetheinformationinputsavailabletothenetworkbyconnectingtwohidden,directionallyopposinglayerstothesameoutput.UsingBRNNs,theoutputlayercangetinformationfrombothpastandfuturestates. BoltzmannMachine(BM) Arecurrentneuralnetwork,thisalgorithmiscapableoflearninginternalrepresentationsandcanrepresentandsolvetoughcombinedproblems. ConvolutionalNeuralNetwork(CNN) Mostcommonlyusedtoanalyzevisualimagery,CNNsareafeed-forwardneuralnetworkdesignedtominimizepre-processing. DeconvolutionalNeuralNetwork(DNN) DNNsenableunsupervisedconstructionofhierarchicalimagerepresentations.Eachlevelofthehierarchygroupsinformationfromtheprecedingleveltoaddmorecomplexfeaturestoanimage. DeepBeliefNetwork(DBN) Whentrainedwithanunsupervisedsetofexamples,aDBNcanlearntoreconstructitsinputsprobabilisticallybyusinglayersasfeaturedetectors.Followingthisprocess,youcantrainaDBNtoperformsupervisedclassifications. DeepConvolutionalInverseGraphicsNetwork(DCIGN) ADCIGNmodelaimstolearnaninterpretablerepresentationofimagesthatthesystemseparatesaccordingtotheelementsofthree-dimensionalscenestructure,suchaslightingvariationsanddepthrotations.ADCIGNusesmanylayersofoperators,bothconvolutionalanddeconvolutional. DeepResidualNetwork(DRN) DRNsassistinhandlingsophisticateddeeplearningtasksandmodels.Byhavingmanylayers,aDRNpreventsthedegradationofresults. DenoisingAutoencoder(DAE) YouuseDAEstoreconstructdatafromcorrupteddatainputs;thealgorithmforcesthehiddenlayertolearnmorerobustfeatures.Asaresult,theoutputyieldsamorerefinedversionoftheinputdata. EchoStateNetwork(ESN) AnESNworkswitharandom,large,fixedrecurrentneuralnetwork,whereineachnodereceivesanonlinearresponsesignal.Thealgorithmrandomlysetsandassignsweightsandconnectivityinordertoattainlearningflexibility. ExtremeLearningMachine(ELM) Thisalgorithmlearnshiddennodeoutputweightingsinonestep,creatingalinearmodel.ELMscangeneralizewellandlearnmanytimesfasterthanbackpropagationnetworks. FeedForwardNeuralNetwork(FForFFNN)andPerceptron(P) Thesearethebasicalgorithmsforneuralnetworks.Afeedforwardneuralnetworkisanartificialneuralnetworkinwhichnodeconnectionsdon’tformacycle;aperceptronisabinaryfunctionwithonlytworesults(up/down;yes/no,0/1). GatedRecurrentUnit(GRU) GRUsuseconnectionsthroughnodesequencestoperformmachinelearningtasksassociatedwithclusteringandmemory.GRUsrefineoutputsthroughthecontrolofmodelinformationflow. GenerativeAdversarialNetwork(GAN) Thissystempitstwoneuralnetworks—discriminativeandgenerative—againsteachother.Theobjectiveistodistinguishbetweenrealandsyntheticresultsinordertosimulatehigh-levelconceptualtasks. HopfieldNetwork(HN) Thisformofrecurrentartificialneuralnetworkisanassociativememorysystemwithbinarythresholdnodes.Designedtoconvergetoalocalminimum,HNsprovideamodelforunderstandinghumanmemory. KohonenNetwork(KN) AKNorganizesaproblemspaceintoatwo-dimensionalmap.Thedifferencebetweenself-organizingmaps(SOMs)andotherproblem-solvingapproachesisthatSOMsusecompetitivelearningratherthanerror-correctionlearning. LiquidStateMachine(LSM) Knownasthird-generationmachinelearning(oraspikingneuralnetwork),anLSMaddstheconceptoftimeasanelement.LSMsgeneratespatiotemporalneuronnetworkactivationastheypreservememoryduringprocessing.PhysicsandcomputationalneuroscienceuseLSMs. Long/Short-TermMemory(LSTM) LSTMiscapableoflearningorrememberingorderdependenceinpredictionproblemsconcerningsequence.AnLSTMunitholdsacell,aninputgate,anoutputgate,andaforgetgate.Cellsretainvaluesoverarbitrarytimeintervals.EachunitregulatesvalueflowsthroughLSTMconnections.Thissequencingcapabilityisessentialincomplexproblemdomains,likespeechrecognitionandmachinetranslation. MarkovChain(MC) AnMCisamathematicalprocessthatdescribesasequenceofpossibleeventsinwhichtheprobabilityofeacheventdependsexclusivelyonthestateattainedinthepreviousevent.Useexamplesincludetyping-wordpredictionsandGooglePageRank. NeuralTuringMachine(NTM) Basedonthemid-20th-centuryworkofdatascientistAlanTuring,anNTMperformscomputationsandextendsthecapabilitiesofneuralnetworksbycouplingwithexternalmemory.DevelopersuseNTMinrobotsandregarditasoneofthemeanstobuildanartificialhumanbrain.   RadialBasisFunctionNetworks(RBFnets) DevelopersuseRBFnetstomodeldatathatrepresentsanunderlyingtrendorfunction.RBFnetslearntoapproximatetheunderlyingtrendusingbellcurvesornon-linearclassifiers.Non-linearclassifiersanalyzemoredeeplythandosimplelinearclassifiersthatworkonlowerdimensionalvectors.Youusethesenetworksinsystemcontrolandtimeseriespredictions. RecurrentNeuralNetwork(RNN) RNNsmodelsequentialinteractionsviamemory.Ateachtimestep,anRNNcalculatesanewmemoryorhiddenstatereliantonboththecurrentinputandpreviousmemorystate.Applicationsincludemusiccomposition,robotcontrol,andhumanactionrecognition. RestrictedBoltzmannMachine(RBM) AnRBMisaprobabilisticgraphicalmodelinanunsupervisedenvironment.AnRBMconsistsofvisibleandhiddenlayersaswellastheconnectionsbetweenbinaryneuronsineachoftheselayers.RBNsareusefulforfiltering,featurelearning,andclassification.Usecasesincluderiskdetectionandbusinessandeconomicanalyses. SupportVectorMachine(SVM) Basedontrainingexamplesetsthatarerelevanttooneoftwopossiblecategories,anSVMalgorithmbuildsamodelthatassignsnewexamplestooneoftwocategories.Themodelthenrepresentstheexamplesasmappedpointsinspacewhiledividingthoseseparatecategoryexamplesbythewidestpossiblegap.Thealgorithmthenmapsnewexamplesinthatsamespaceandpredictswhatcategorytheybelongtobasedonwhichsideofthegaptheyoccupy.Applicationsincludefacedetectionandbioinformatics. VariationalAutoencoder(VAE) AVAEisaspecifictypeofneuralnetworkthathelpsgeneratecomplexmodelsbasedondatasets.Ingeneral,anautoencoderisadeeplearningnetworkthatattemptstoreconstructamodelormatchthetargetoutputstoprovidedinputsthroughbackpropagation.AVAEalsoyieldsstate-of-the-artmachinelearningresultsintheareasofimagegenerationandreinforcementlearning. WhatAreNeuralNetworksinDataMining? Inherpaper“NeuralNetworksinDataMining,”PriyankaGuarnotesthat,“Inmorepracticalterms,neuralnetworksarenon-linearstatisticaldatamodelingtools.Theycanbeusedtomodelcomplexrelationshipsbetweeninputsandoutputsortofindpatternsindata.Usingneuralnetworksasatool,datawarehousingfirmsareharvestinginformationfromdatasetsintheprocessknownasdatamining.”   Gaurcontinues,“Thedifferencebetweenthesedatawarehousesandordinarydatabasesisthatthereisactualmanipulationandcross-fertilizationofthedata,helpingusersmakemoreinformeddecisions.” Althoughyoucanuseneuralnetworkstodatamine,developersgenerallydon’tbecauseNNsrequirelongtrainingtimesandoftenproducehard-to-comprehendmodels.Whenprofessionalsdodecidetousethem,theyhavetwotypesofneuralnetworkdataminingapproachestochoosefrom:onedirectlylearnssimple,easy-to-understandnetworks,whiletheotheremploysthemorecomplicatedruleextraction,whichinvolvesextractingsymbolicmodelsfromtrainedneuralnetworks. Neuralvs.ConventionalComputers Oneoftheprimarydifferencesbetweenconventional,ortraditional,computersandneuralcomputersisthatconventionalmachinesprocessdatasequentially,whileneuralnetworkscandomanythingsatonce.Herearesomeoftheothermajordifferencesbetweenconventionalandneuralcomputers: FollowingInstructionsvs.LearningCapability:Conventionalcomputerslearnonlybyperformingstepsorsequencessetbyanalgorithm,whileneuralnetworkscontinuouslyadapttheirprogrammingandessentiallyprogramthemselvestofindsolutions.Conventionalcomputersarelimitedbytheirdesign,whileneuralnetworksaredesignedtosurpasstheiroriginalstate. Rulesvs.ConceptsandImagery:Conventionalcomputersoperatethroughlogicfunctionsbasedonagivensetofrulesandcalculations.Incontrast,artificialneuralnetworkscanrunthroughlogicfunctionsanduseabstractconcepts,graphics,andphotographs.Traditionalcomputersarerules-based,whileartificialneuralnetworksperformtasksandthenlearnfromthem.  Complementary,NotEqual:Conventionalalgorithmiccomputersandneuralnetworkscomplementeachother.Sometasksaremorearithmeticallybasedanddon’trequirethelearningabilityofneuralnetworks.Oftenthough,tasksrequirethecapabilitiesofbothsystems.Inthesecases,theconventionalcomputersupervisestheneuralnetworkforhigherspeedandefficiency. Asimpressiveasneuralnetworksare,they’restillworks-in-progress,presentingchallengesaswellaspromiseforthefutureofproblem-solving. TheChallengesofNeuralNetworks Cortx’sCardinellsaysthatthevalueandimplementationofneuralnetworksdependonthetask,soit’simportanttounderstandthechallengesandlimitations:“Ourgeneralapproachistodowhatworksforeachspecificproblemwe’retryingtosolve.Inmanyofthosecases,thatinvolvesusingneuralnetworks;inothercases,weusemoretraditionalapproaches.”Cardinellillustrateshispointwiththisexample:“Forinstance,inPerfectTense,wetrytodetectwhethersomeoneisusingaorancorrectly.Inthiscase,usinganeuralnetworkwouldbeoverkill,becauseyoucansimplylookatthephoneticpronunciationtomakethedetermination(e.g.,anbananaiswrong).Neuralnetworksarewheremostadvancesarebeingmaderightnow.Thingsthatwereimpossibleonlyayearortwoagoregardingcontentqualityarenowareality.” Asusefulasneuralnetworkscanbe,challengesinthefieldabound: Training:Acommoncriticismofneuralnetworks,particularlyinroboticsapplications,isthatexcessivetrainingforreal-worldoperationsismandatory.Onewaytoovercomethathurdleisbyrandomlyshufflingtrainingexamples.Usinganumericaloptimizationalgorithm,smallsteps—ratherthanlargesteps—aretakentofollowanexample.Anotherwayisbygroupingexamplesinso-calledmini-batches.Improvingtrainingefficienciesandconvergencecapabilitiesisanongoingresearchareaforcomputerscientists. TheoreticalIssues:Unsolvedproblemsremain,evenforthemostsophisticatedneuralnetworks.Forexample,despiteitsbestefforts,Facebookstillfindsitimpossibletoidentifyallhatespeechandmisinformationbyusingalgorithms.Thecompanyemploysthousandsofhumanreviewerstoresolvetheproblem.Ingeneral,becausecomputersaren’thuman,theirabilitytobegenuinelycreative—provemaththeorems,makemoralchoices,composeoriginalmusic,ordeeplyinnovate—isbeyondthescopeofneuralnetworksandAI. Inauthenticity:Thetheoreticalchallengesweaddressabovearisebecauseneuralnetworksdon’tfunctionexactlyashumanbrainsdo—theyoperatemerelyasasimulacrumofthehumanbrain.Thespecificsofhowmammalianneuronscodeinformationisstillanunknown.Artificialneuralnetworksdon’tstrictlyreplicateneuralfunction,butratherusebiologicalneuralnetworksastheirinspiration.Thisprocessallowsstatisticalassociation,whichisthebasisofartificialneuralnetworks.AnANN’slearningprocessisn’tidenticaltothatofahuman,thus,itsinherent(atleastfornow)limitations. HardwareIssues:Thiscentury’sfocusonneuralnetworksisduetothemillion-foldincreaseincomputingpowersince1991.Morehardwarecapacityhasenabledgreatermulti-layeringandsubsequentdeeplearning,andtheuseofparallelgraphicsprocessingunits(GPUs)nowreducestrainingtimesfrommonthstodays.DespitethegreatstridesofNNsinveryrecentyears,asdeepneuralnetworksmature,developersneedhardwareinnovationstomeetincreasingcomputationaldemands.Thesearchison,andnewdevicesandchipsdesignedspecificallyforAIareindevelopment.A2018NewYorkTimesarticle,“BigBetsonA.I.OpenaNewFrontierforChipStartups,Too,”reportedthat“venturecapitalistsinvestedmorethan$1.5billioninchipstartups”in2017. Hybrids:AproposaltoovercomesomeofthechallengesofneuralnetworkscombinesNNwithsymbolicAI,orhuman-readablerepresentationsofsearch,logic,andproblems.Tosuccessfullyduplicatehumanintelligence,it’svitaltotranslatetheproceduralknowledgeorimplicitknowledge(theskillsandknowledgenotreadilyaccessiblebyconsciousawareness)humanspossessintoanunequivocalformthatusessymbolsandrules.Sofar,thedifficultiesofdevelopingsymbolicAIhavebeenunresolvable—butthatstatusmaysoonchange. Computerscientistsareworkingtoeliminatethesechallenges.LeadersinthefieldofneuralnetworksandAIarewritingsmarter,faster,morehumanalgorithmseveryday.Engineersaredrivingimprovementsbyusingbetterhardwareandcross-pollinatingdifferenthardwareandsoftware TheFutureofNeuralNetworks   “Weneedtorememberthatartificialneuralnetworksanddeeplearningarebutonesetoftechniquesfordevelopingsolutionstospecificproblems.Rightnow,they’rethe‘bigthing,’”opinesRichardYonck,FounderandLeadFuturistofIntelligentFutureConsultingandauthorofHeartoftheMachine:OurFutureinaWorldofArtificialEmotionalIntelligence. Headds,“It’sthatoldsaying:‘Whenyouronlytoolisahammer,everythinglookslikeanail.’Excepteverythingisn’tanail,anddeeplearningdoesn’tworkforallproblems.Thereareallsortsofdevelopmentstocomeinthenextcoupleofdecadesthatmayprovidebettersolutions:one-shotlearning,contextualnaturallanguageprocessing,emotionengines,commonsenseengines,andartificialcreativity.” Herearesomelikelyfuturedevelopmentsinneuralnetworktechnologies: FuzzyLogicIntegration:Fuzzylogicrecognizesmorethansimpletrueandfalsevalues—ittakesintoaccountconceptsthatarerelative,likesomewhat,sometimes,andusually.Fuzzylogicandneuralnetworksareintegratedforusesasdiverseasscreeningjobapplicants,auto-engineering,buildingcranecontrol,andmonitoringglaucoma.Fuzzylogicwillbeanessentialfeatureinfutureneuralnetworkapplications. PulsedNeuralNetworks:Recently,neurobiologicalexperimentdatahasclarifiedthatmammalianbiologicalneuralnetworksconnectandcommunicatethroughpulsingandusethetimingofpulsestotransmitinformationandperformcomputations.Thisrecognitionhasacceleratedsignificantresearch,includingtheoreticalanalyses,modeldevelopment,neurobiologicalmodeling,andhardwaredeployment,allaimedatmakingcomputingevenmoresimilartothewayourbrainsfunction. SpecializedHardware:There’scurrentlyadevelopmentexplosiontocreatethehardwarethatwillspeedandultimatelylowerthepriceofneuralnetworks,machinelearning,anddeeplearning.Establishedcompaniesandstartupsareracingtodevelopimprovedchipsandgraphicprocessingunits,buttherealnewsisthefastdevelopmentofneuralnetworkprocessingunits(NNPUs)andotherAIspecifichardware,collectivelyreferredtoasneurosynapticarchitectures.NeurosynapticchipsarefundamentaltotheprogressofAIbecausetheyfunctionmorelikeabiologicalbrainthanthecoreofatraditionalcomputer.WithitsBrainPowertechnology,IBMhasbeenaleaderinthedevelopmentofneurosynapticchips.Unlikestandardchips,whichruncontinuously,BrainPower’schipsareevent-drivenandoperateonanas-neededbasis.Thetechnologyintegratesmemory,computation,andcommunication. ImprovementofExistingTechnologies:Enabledbynewsoftwareandhardwareaswellasbycurrentneuralnetworktechnologiesandtheincreasedcomputingpowerofneurosynapticarchitectures,neuralnetworkshaveonlybeguntoshowwhattheycando.Themyriadbusinessapplicationsoffaster,cheaper,andmorehuman-likeproblem-solvingandimprovedtrainingmethodsarehighlylucrative.  Robotics:Therehavebeencountlesspredictionsaboutrobotsthatwillbeabletofeellikeus,seelikeus,andmakeprognosticationsabouttheworldaroundthem.Thesepropheciesevenincludesomedystopianversionsofthatfuture,fromtheTerminatorfilmseriestoBladeRunnerandWestworld.However,futuristYoncksaysthatwestillhaveaverylongwaytogobeforerobotsreplaceus:“Whiletheserobotsarelearninginalimitedway,it’saprettyfarleaptosaythey’re‘thinking.’Therearesomanythingsthathavetohappenbeforethesesystemscantrulythinkinafluid,non-brittleway.OneofthecriticalfactorsIbringupinmybookistheabilitytoestablishandactonself-determinedvaluesinreal-time,whichwehumansdothousandsoftimesaday.Withoutthis,thesesystemswillfaileverytimeconditionsfalloutsideapredefineddomain.” Mind-meldingbetweenhumanandartificialbrains,accordingtoYonck,isinourfuture:“Ithinkartificialintelligence,artificialneuralnetworks,anddeeplearningwilleventuallyplayafarmoreactiveroleinretrainingourbrains,particularlyasbrain-computerinterfaces(BCIs)becomemoreprevalentandwidelyused.Deeplearningwillbeessentialforlearningtoreadandinterpretanindividualbrain’slanguage,anditwillbeusedtooptimizeadifferentaspectofthought—focus,analysis,introspection.Eventually,thismaybethepathtoIA(intelligenceaugmentation),aformofblendedintelligencewe’llseearoundthemiddleofthiscentury.” ResourcesonNeuralNetworks Thebravenewworldofneuralnetworkscanbehardtounderstandandisconstantlychanging,sotakeadvantageoftheseresourcestostayabreastofthelatestdevelopments. Neuralnetworkassociationssponsorconferences,publishpapersandperiodicals,andpostthelatestdiscoveriesabouttheoryandapplications.BelowisalistofsomeofthemajorNNassociationsandhowtheydescribetheirorganizationalgoals: TheInternationalNeuralNetworkSociety(INNS):Theorganizationisfor“individualsinterestedinatheoreticalandcomputationalunderstandingofthebrainandapplyingthatknowledgetodevelopnewandmoreeffectiveformsofmachineintelligence.” IEEEComputationalIntelligenceSociety(IEEECIS):ThisisaprofessionalsocietyoftheInstituteofElectricalandElectronicsEngineers(IEEE)whofocuson“thetheory,design,application,anddevelopmentofbiologicallyandlinguisticallymotivatedcomputationalparadigmsthatemphasizetheneuralnetworks,connectionistsystems,geneticalgorithms,evolutionaryprogramming,fuzzysystems,andhybridintelligentsystemsinwhichtheseparadigmsarecontained.” EuropeanNeuralNetworkSociety(ENNS):Thisisan“associationofscientists,engineers,students,andothersseekingtolearnaboutandadvanceourunderstandingofthemodelingofbehavioralandbrainprocesses,developneuralalgorithms,andapplyneuralmodelingconceptstoproblemsrelevantinmanydifferentdomains.” InternationalInstituteforForecasters(IIF):Thisorganizationis“dedicatedtodevelopingandfurtheringthegeneration,distribution,anduseofknowledgeonforecasting.” Mostofthetitlesprovidedbelowhavebeenpublishedwithinthelasttwoyears.We’vealsoincludedafewclassicsofthediscipline: Aggarwal,CharuC.NeuralNetworksandDeepLearning:ATextbook.NewYorkCity:SpringerInternationalPublishing,2018. Goldberg,Yoav.NeuralNetworkMethodsforNaturalLanguageProcessing(SynthesisLecturesonHumanLanguageTechnologies).Williston:Morgan&ClaypoolPublishers,2017. Hagan,MartinT.,Demuth,HowardB.,andBeale,MarkH.NeuralNetworkDesign(2ndEdition).MartinHagan,2014.  Hassoun,Mohamad.FundamentalsofArtificialNeuralNetworks.Cambridge:TheMITPress|ABradfordBook,2013. Haykin,SimonO.NeuralNetworksandLearningMachines(3rdEdition).Chennai:PearsonIndia,2008. Heaton,Jeff.IntroductiontotheMathofNeuralNetworks.HeatonResearch,Inc.,2012. Taylor,Michael.MakeYourOwnNeuralNetwork:AnIn-DepthVisualIntroductionforBeginners.IndependentlyPublished,2017. Theworldofneuralnetworkshasitsownlanguage.Herearesomeresourcestoexpandyourtechnicalvocabularyandunderstandingofthefield: ESANeuralNetworkGlossary:AcompilationofneuralnetworkingtermsfromtheEuropeanSpaceAgencies’EarthnetOnlinesite MediumNeuralNetworkGlossary:Afrequentlyupdatedlistofthelatestterminologyfromthetechwritingsourcesite,Medium SkymindA.I.WikiGlossary:Afrequentlyupdatedcompendiumofclearlydefinedtermsconcerningneuralnetworksanddeepartificialnetworks TheFutureofWorkwithAutomatedProcessesinSmartsheet Empoweryourpeopletogoaboveandbeyondwithaflexibleplatformdesignedtomatchtheneedsofyourteam—andadaptasthoseneedschange.  TheSmartsheetplatformmakesiteasytoplan,capture,manage,andreportonworkfromanywhere,helpingyourteambemoreeffectiveandgetmoredone.Reportonkeymetricsandgetreal-timevisibilityintoworkasithappenswithroll-upreports,dashboards,andautomatedworkflowsbuilttokeepyourteamconnectedandinformed.  Whenteamshaveclarityintotheworkgettingdone,there’snotellinghowmuchmoretheycanaccomplishinthesameamountoftime. TrySmartsheetforfree,today.     Discoverwhyover90%ofFortune100companiestrustSmartsheettogetworkdone. TrySmartsheetforFree GetaFreeSmartsheetDemo