| 词条 | Cascading (software) |
| 释义 |
| name = Cascading | status = Active | latest release version = 3.1 | programming language = Java | license = Apache License | website = http://www.cascading.org/ }}Cascading is a software abstraction layer for Apache Hadoop and Apache Flink. Cascading is used to create and execute complex data processing workflows on a Hadoop cluster using any JVM-based language (Java, JRuby, Clojure, etc.), hiding the underlying complexity of MapReduce jobs. It is open source and available under the Apache License. Commercial support is available from Driven, Inc.[1] Cascading was originally authored by Chris Wensel, who later founded Concurrent, Inc, which has been re-branded as Driven.[2] Cascading is being actively developed by the community{{citation needed|date=October 2014}} and a number of add-on modules are available.[3] ArchitectureTo use Cascading, Apache Hadoop must also be installed, and the Hadoop job .jar must contain the Cascading .jars. Cascading consists of a data processing API, integration API, process planner and process scheduler. Cascading leverages the scalability of Hadoop but abstracts standard data processing operations away from underlying map and reduce tasks.[4]{{Better source|date=October 2013}} Developers use Cascading to create a .jar file that describes the required processes. It follows a ‘source-pipe-sink’ paradigm, where data is captured from sources, follows reusable ‘pipes’ that perform data analysis processes, where the results are stored in output files or ‘sinks’. Pipes are created independent from the data they will process. Once tied to data sources and sinks, it is called a ‘flow’. These flows can be grouped into a ‘cascade’, and the process scheduler will ensure a given flow does not execute until all its dependencies are satisfied. Pipes and flows can be reused and reordered to support different business needs.[5] Developers write the code in a JVM-based language and do not need to learn MapReduce. The resulting program can be regression tested and integrated with external applications like any other Java application.[6] Cascading is most often used for ad targeting, log file analysis, bioinformatics, machine learning, predictive analytics, web content mining, and extract, transform and load (ETL) applications.[7] Uses of CascadingCascading was cited as one of the top five most powerful Hadoop projects by SD Times in 2011,[8]{{Unreliable source?|date=October 2013}} as a major open source project relevant to bioinformatics[9]{{Unreliable source?|date=October 2013}} and is included in Hadoop: A Definitive Guide, by Tom White.[10] The project has also been cited in presentations, conference proceedings and Hadoop user group meetings as a useful tool for working with Hadoop[11][12][13][14] and with Apache Spark[15]
Domain-Specific Languages Built on Cascading
References1. ^Cascading support page 2. ^Driven, Inc. 3. ^Cascading modules 4. ^Blog post by Etsy describing their use of Cascading with Hadoop 5. ^Cascading User Guide {{webarchive |url=https://web.archive.org/web/20110206053054/http://www.cascading.org/1.2/userguide/pdf/userguide.pdf |date=February 6, 2011 }} 6. ^Driven product page 7. ^Driven home page 8. ^{{cite news| last = Handy| first = Alex| date = 1 June 2011| title = The top five most powerful Hadoop projects| url =http://www.sdtimes.com/content/article.aspx?ArticleID=35596&page=1| newspaper = SD Times| location = | publisher = | accessdate = 26 October 2013}} 9. ^{{cite news| last = Taylor| first = Ronald| date = 21 December 2010| title = An overview of the Hadoop/MapReduce/HBase framework and its current applications in bioinformatics| url = http://www.biomedcentral.com/1471-2105/11/S12/S1| newspaper = BioMed Central| location = | publisher = Springer Science+Business Media| accessdate = 26 October 2013}} 10. ^[https://books.google.com/books?id=Nff49D7vnJcC&lpg=PA539&dq=cascading%20hadoop&pg=PA548#v=onepage&q=cascading%20hadoop&f=false White, Tom, “Hadoop: The Definitive Guide,” O’Reilly Media, Inc., 2010, pp. 539 – 549.] 11. ^Nathan, Paco (Wikipedia: Paco Nathan), “Getting Started on Hadoop” presentation for the SV Cloud Computing Meetup, 7/19/2010. 12. ^Julio Guijarro, Steve Loughran and Paolo Castagna, “Hadoop and beyond,” HP Labs, Bristol UK, 2008. 13. ^Cross, Bradford, “Flightcaster_HUG,” Presentation at the Bay Area Hadoop Users’ Group, March 26, 2010 14. ^Curtin, Christopher, “NoSQL, Hadoop and Cascading,” June 2010. 15. ^{{Cite web|url=https://spark-summit.org/2014/talk/using-cascading-to-build-data-centric-applications-on-spark|title=Using Cascading to Build Data-centric Applications on Spark|date=2014-05-07|website=Spark Summit 2014|access-date=2016-03-25}} 16. ^Cascading{{Not a typo|.}}Multitool on AWS 17. ^LogAnalyzer for Amazon CloudFront 18. ^BackType blog {{webarchive |url=https://web.archive.org/web/20110825014616/http://tech.backtype.com/ |date=August 25, 2011 }} 19. ^Blog post by Etsy describing their use of Cascading with Hadoop 20. ^FlightCaster 21. ^Ion Flux {{webarchive |url=https://web.archive.org/web/20111023203553/http://www.concurrentinc.com/casestudies/ion_flux |date=October 23, 2011 }} 22. ^RapLeaf Blog {{webarchive |url=https://web.archive.org/web/20110201023302/http://blog.rapleaf.com/dev/2008/09/05/goodbye-mapreduce-hello-cascading/ |date=February 1, 2011 }} 23. ^Razorfish 24. ^[https://github.com/twitter/pycascading] 25. ^[https://github.com/gmarabout/cascading.jruby Cascading.jruby] 26. ^[https://github.com/nathanmarz/cascalog Cascalog] 27. ^[https://github.com/twitter/scalding Scalding] External links
4 : Free software programmed in Java (programming language)|Free system software|Cloud infrastructure|Hadoop |
| 随便看 |
|
开放百科全书收录14589846条英语、德语、日语等多语种百科知识，基本涵盖了大多数领域的百科知识，是一部内容自由、开放的电子版国际百科全书。