A library of books processed through stream pipelines
The complete source of streamsAndPipelines_library.ek9, compiled by the EK9 compiler. Shown on streamsAndPipelines.html.
#!ek9 defines module site.streamsAndPipelines.library defines constant MINIMUM_BOOKS <- 3 SENIOR_AGE <- 50 defines type AuthorId as Integer constrain > 0 Age as Integer constrain > 0 Name as String constrain as matches /^[a-zA-Z -]+$/ BookTitle as String constrain as matches /^[a-zA-Z0-9 -+]+$/ <?- Simple enumeration for controlling filtering -?> BookFilterSelection SkipTwo JustFirst JustLast defines class <?- Models the author of books. -?> Author id as AuthorId? age as Age? firstname as Name? surname as Name? Author() as pure -> id as AuthorId age as Age firstname as Name surname as Name require id? and age? and firstname? and surname? this.id:=? AuthorId(id) this.age:=? Age(age) this.firstname:=? Name(firstname) this.surname:=? Name(surname) Author() as pure -> author as Author this(id: author.id(), age: author.age(), firstname: author.firstname(), surname: author.surname()) <?- Stop creation of empty Author. -?> default private Author() as pure id() as pure <- rtn as AuthorId: id age() as pure <- rtn as Age: age firstname() as pure <- rtn as Name: firstname surname() as pure <- rtn as Name: surname operator == as pure -> author as Author <- rtn as Boolean: id == author.id() operator #? as pure <- rtn as Integer: #?id operator $ as pure <- rtn as String: `Author: ${firstname} ${surname}, Age: ${age}` override operator ? as pure <- rtn as Boolean: age? and firstname? and surname? operator :=: -> author as Author id :=: author.id() age :=: author.age() firstname :=: author.firstname() surname :=: author.surname() Book title as BookTitle? author as Author? published as Date? Book() as pure -> title as BookTitle author as Author published as Date require title? and author? and published? this.title:=? BookTitle(title) this.author:=? Author(author) this.published:=? Date(published) <?- Stop creation of empty Book. -?> default private Book() as pure author() as pure <- rtn as Author: author published() as pure <- rtn as Date: published operator $ as pure <- rtn as String: `Title: ${title}, ${author}, Published: ${published}` operator #? as pure <- rtn as Integer: #? $this operator #^ as pure <- rtn as String: $this operator <=> as pure -> book as Book <- rtn as Integer: published() <=> book.published() override operator ? as pure <- rtn as Boolean: title? and author? and published? <?- Model the concept of a library, just an in-memory map for now -?> Library books as List of Book? Library() as pure -> withBooks as List of Book books :=? withBooks Library() as pure this([ Book(BookTitle("Java"), Author(id: AuthorId(1), age: Age(50), firstname: Name("John"), surname: Name("Doe")), 1998-01-01), Book(BookTitle("C++"), Author(id: AuthorId(1), age: Age(42), firstname: Name("John"), surname: Name("Doe")), 1990-01-07), Book(BookTitle("Scala"), Author(id: AuthorId(1), age: Age(67), firstname: Name("John"), surname: Name("Doe")), 2015-03-02), Book(BookTitle("Python"), Author(id: AuthorId(1), age: Age(62), firstname: Name("John"), surname: Name("Doe")), 2010-12-02), Book(BookTitle("HTML"), Author(id: AuthorId(2), age: Age(58), firstname: Name("Mark"), surname: Name("Pickford")), 2008-07-02), Book(BookTitle("CSS"), Author(id: AuthorId(4), age: Age(51), firstname: Name("Mark"), surname: Name("Keely")), 2008-04-02), Book(BookTitle("ADA"), Author(id: AuthorId(5), age: Age(44), firstname: Name("Ada"), surname: Name("Lovelace")), 1988-01-02), Book(BookTitle("Dart"), Author(id: AuthorId(6), age: Age(47), firstname: Name("Peter"), surname: Name("Dove")), 2020-01-02), Book(BookTitle("C#"), Author(id: AuthorId(7), age: Age(60), firstname: Name("William"), surname: Name("Fence")), 2012-10-02), Book(BookTitle("Javascript"), Author(id: AuthorId(3), age: Age(52), firstname: Name("James"), surname: Name("Pickford")), 2008-03-02), Book(BookTitle("C"), Author(id: AuthorId(1), age: Age(42), firstname: Name("John"), surname: Name("Doe")), 1990-01-07), Book(BookTitle("C++"), Author(id: AuthorId(7), age: Age(38), firstname: Name("William"), surname: Name("Fence")), 1990-04-02), Book(BookTitle("C"), Author(id: AuthorId(7), age: Age(38), firstname: Name("William"), surname: Name("Fence")), 1990-04-14), Book(BookTitle("Haskell"), Author(id: AuthorId(7), age: Age(30), firstname: Name("William"), surname: Name("Fence")), 1982-04-14), Book(BookTitle("Lisp"), Author(id: AuthorId(7), age: Age(25), firstname: Name("William"), surname: Name("Fence")), 1977-09-24) ]) <?- While you would never really 'iterate' through all the books in a library - this just enables the demonstration of streaming. Clearly you would normally have multiple maps/indexes etc to locate books via a search mechanism. -?> iterator() as pure <- rtn as Iterator of Book: books.iterator() operator + as pure -> book as Book <- rtn as Library: Library(books + book) override operator ? as pure <- rtn as Boolean: books? defines program <?- Just send all the books in the library out to stdout. Because Library has an iterator method 'cat' will use it to get each book. -?> JustCatBooks() stdout <- Stdout() library <- Library() cat library > stdout defines function comparingAuthor() as pure -> book1 as Book book2 as Book <- rtn as Integer: book1.author().id() <=> book2.author().id() defines program <?- Now sort the books before outputting. This uses a specific comparator. -?> SortBooksByAuthor() stdout <- Stdout() library <- Library() cat library | sort by comparingAuthor > stdout defines function dateBookPublished() as pure -> book as Book <- rtn as Date: book.published() comparingAuthorThenDate() as pure -> book1 as Book book2 as Book <- rtn as Integer: comparingAuthor(book1, book2) if rtn == 0 rtn: book1 <=> book2 defines program <?- Example of grouping books by the same author that were published on the same date. 'group' only joins ADJACENT books whose keys are equal, so the books are first sorted by author and then by published date; each run of equal dates then becomes one group. The groups are output in the order they were created, each as a List of Book on its own line. -?> GroupBooksByPublishedDate() stdout <- Stdout() library <- Library() cat library | sort by comparingAuthorThenDate | group by dateBookPublished > stdout defines function compareDatePublished() as pure -> book1 as Book book2 as Book <- rtn as Integer: book1.published() <=> book2.published() orderOnPublishedDate() as pure -> books as List of Book <- rtn as List of Book: cat books | sort by compareDatePublished | collect as List of Book defines program <?- This example outputs the books in author order, but also for each of those authors sorts their books by published date. -?> ProcessByAuthor() stdout <- Stdout() library <- Library() //This is the nearest ek9 gets to a lambda - it's a bit 'wordy'. //In effect we are creating a function delegate that is a 'Function' that accepts a 'Book' and //returns an 'AuthorId' type. It is pure in nature - it does not mutate the book in anyway. //The arguments of the 'Function' inferred from its definition and so can just be used in the parenthesis. authorId <- () is Function of (Book, AuthorId) as pure function (r:=? t.author().id()) cat library | sort by comparingAuthor | group by authorId | map by orderOnPublishedDate | flatten > stdout defines function sufficientBooks() as pure -> books as List of Book <- rtn as Boolean: length books >= MINIMUM_BOOKS defines program <?- This example shows two things, firstly it inlines the dynamic function (not keen on this but it can be done). Then it filters the lists that do not have sufficient books (in this case three or more). Only then does it order those books on their published date. -?> ProcessByAuthorWithThreeOrMoreBooks() stdout <- Stdout() library <- Library() cat library | sort by comparingAuthor | group by () is Function of (Book, AuthorId) as pure function (r:=? t.author().id()) | filter with sufficientBooks | map by orderOnPublishedDate | flatten > stdout defines program <?- Similar to the other examples, but this one just skips the first two books by each author (if they have 3 or more books). Note, there is a sort of common pattern to these, if an intermediate operation needs a function, you can either define a basic function or just create a dynamic function. -?> ProcessByAuthorWithThreeOrMoreBooksIgnoreFirstTwo() stdout <- Stdout() library <- Library() authorId <- () is Function of (Book, AuthorId) as pure function (r:=? t.author().id()) excludingFirstTwoBooks <- () is Function of(List of Book, List of Book) as pure function r:=? cat t | skip 2 | collect as List of Book cat library | sort by comparingAuthor | group by authorId | filter with sufficientBooks | map by orderOnPublishedDate | map by excludingFirstTwoBooks | flatten > stdout defines function bookFilter() as pure abstract -> books as List of Book <- filtered as List of Book? <?- Example of a higher order function and dynamic functions. There are other ways to solve the mapping of an enumerations requirement to a function. This uses a switch/given expression in a pure context -?> suitableBookFilter() as pure -> selection as BookFilterSelection <- rtn as bookFilter: given selection <- theFilter as bookFilter? when BookFilterSelection.SkipTwo theFilter:=? () is bookFilter as pure function (filtered:=? cat books | skip 2 | collect as List of Book) when BookFilterSelection.JustFirst theFilter:=? () is bookFilter as pure function (filtered:=? cat books | head 1 | collect as List of Book) when BookFilterSelection.JustLast theFilter:=? () is bookFilter as pure function (filtered:=? cat books | tail 1 | collect as List of Book) default theFilter:=? () is bookFilter as pure function (filtered:=? books) defines program <?- Example of using a higher order function to create the book filter. -?> ProcessByAuthorUsingHigherOrderFunction() stdout <- Stdout() library <- Library() authorId <- () is Function of (Book, AuthorId) as pure function (r:=? t.author().id()) stdout.println("Omit first two books where author has three or more books") cat library | sort by comparingAuthor | group by authorId | filter with sufficientBooks | map by orderOnPublishedDate | map with suitableBookFilter(BookFilterSelection.SkipTwo) | flatten > stdout defines function filterBooksToOutput() -> library as Library filterSelection as BookFilterSelection output as StringOutput authorId <- () is Function of (Book, AuthorId) as pure function (r:=? t.author().id()) cat library | sort by comparingAuthor | group by authorId | filter with sufficientBooks | map by orderOnPublishedDate | map with suitableBookFilter(filterSelection) | flatten > output defines program <?- Next logical step is to extract the above and put it into a function called 'filterBooksToOutput' Then it can just be called with appropriate parameters. -?> ProcessByAuthorUsingParameterisedFunction() stdout <- Stdout() library <- Library() stdout.println("Omit first two books where author has three or more books") filterBooksToOutput(library, BookFilterSelection.SkipTwo, stdout) stdout.println("First book where author has three or more books") filterBooksToOutput(library, BookFilterSelection.JustFirst, stdout) stdout.println("Last book where author has three or more books") filterBooksToOutput(library, BookFilterSelection.JustLast, stdout) defines function bookSigningEvent() as pure -> book as Book <- rtn as String: `Date: ${book.published()}` defines program UniquePublishingDatesFromAuthorWithThreeOrMoreBooks() stdout <- Stdout() library <- Library() authorId <- () is Function of (Book, AuthorId) as pure function (r:=? t.author().id()) authorsBooks as List of Book: List() booksByEachAuthor as List of List of Book: List() stdout.println("Unique signing events on day of publication by authors with three or more books.") cat library | sort by comparingAuthor | group by authorId | tee in booksByEachAuthor | filter by sufficientBooks | map by orderOnPublishedDate | flatten | tee in authorsBooks | uniq by dateBookPublished | sort by compareDatePublished | map with bookSigningEvent > stdout stdout.println("Books From author with three or more books") cat authorsBooks > stdout stdout.println(`There are ${length booksByEachAuthor} authors in total with any number of books`) defines function bookAuthor() as pure -> book as Book <- author as Author: book.author() acceptableAuthorAge() as pure -> author as Author <- rtn as Boolean: author.age() >= Age(50) authorName() as pure -> author as Author <- rtn as String: `${author.firstname()} ${author.surname()}` comparingAuthorName() as pure -> author1 as Author author2 as Author <- rtn as Integer: author1.surname() <=> author2.surname() if rtn == 0 rtn: author1.firstname() <=> author2.firstname() defines program LibraryExample() stdout <- Stdout() library <- Library() cat library | map with bookAuthor | filter by acceptableAuthorAge | uniq | sort by comparingAuthorName | map with authorName > stdout defines function isModernBook() as pure -> book as Book <- rtn as Boolean: book.published() >= 2000-01-01 firstBookOnly() as pure -> books as List of Book <- rtn as List of Book: cat books | head 1 | collect as List of Book defines program <?- Split books by publication date: every modern book (published 2000 or later) starts a new group, then take just the first book from each group. -?> SplitBooksByEra() stdout <- Stdout() library <- Library() stdout.println("First book from each group, a new group starting at each modern book") cat library | sort by compareDatePublished | split with isModernBook | map by firstBookOnly | flatten > stdout <?- To get exactly two eras, group by era instead: once sorted by date the pre-2000 books and the modern books are each one run of equal keys. -?> GroupBooksByEra() stdout <- Stdout() library <- Library() stdout.println("First book from each era (pre-2000 and 2000+)") cat library | sort by compareDatePublished | group by isModernBook | map by firstBookOnly | flatten > stdout defines program ImperativeLibraryExample() stdout <- Stdout() library <- Library() uniqueAuthors as Dict of (AuthorId, Author): Dict() for book in library author <- book.author() if author.age() >= SENIOR_AGE if uniqueAuthors not contains author.id() uniqueAuthors += DictEntry(author.id(), author) for author in uniqueAuthors.values() stdout.println(`${author.firstname()} ${author.surname()}`)
Download streamsAndPipelines_library.ek9
Output
JustCatBooks
Title: Java, Author: John Doe, Age: 50, Published: 1998-01-01 Title: C++, Author: John Doe, Age: 42, Published: 1990-01-07 Title: Scala, Author: John Doe, Age: 67, Published: 2015-03-02 Title: Python, Author: John Doe, Age: 62, Published: 2010-12-02 Title: HTML, Author: Mark Pickford, Age: 58, Published: 2008-07-02 Title: CSS, Author: Mark Keely, Age: 51, Published: 2008-04-02 Title: ADA, Author: Ada Lovelace, Age: 44, Published: 1988-01-02 Title: Dart, Author: Peter Dove, Age: 47, Published: 2020-01-02 Title: C#, Author: William Fence, Age: 60, Published: 2012-10-02 Title: Javascript, Author: James Pickford, Age: 52, Published: 2008-03-02 Title: C, Author: John Doe, Age: 42, Published: 1990-01-07 Title: C++, Author: William Fence, Age: 38, Published: 1990-04-02 Title: C, Author: William Fence, Age: 38, Published: 1990-04-14 Title: Haskell, Author: William Fence, Age: 30, Published: 1982-04-14 Title: Lisp, Author: William Fence, Age: 25, Published: 1977-09-24
SortBooksByAuthor
Title: Java, Author: John Doe, Age: 50, Published: 1998-01-01 Title: C++, Author: John Doe, Age: 42, Published: 1990-01-07 Title: Scala, Author: John Doe, Age: 67, Published: 2015-03-02 Title: Python, Author: John Doe, Age: 62, Published: 2010-12-02 Title: C, Author: John Doe, Age: 42, Published: 1990-01-07 Title: HTML, Author: Mark Pickford, Age: 58, Published: 2008-07-02 Title: Javascript, Author: James Pickford, Age: 52, Published: 2008-03-02 Title: CSS, Author: Mark Keely, Age: 51, Published: 2008-04-02 Title: ADA, Author: Ada Lovelace, Age: 44, Published: 1988-01-02 Title: Dart, Author: Peter Dove, Age: 47, Published: 2020-01-02 Title: C#, Author: William Fence, Age: 60, Published: 2012-10-02 Title: C++, Author: William Fence, Age: 38, Published: 1990-04-02 Title: C, Author: William Fence, Age: 38, Published: 1990-04-14 Title: Haskell, Author: William Fence, Age: 30, Published: 1982-04-14 Title: Lisp, Author: William Fence, Age: 25, Published: 1977-09-24
GroupBooksByPublishedDate
[Title: C++, Author: John Doe, Age: 42, Published: 1990-01-07, Title: C, Author: John Doe, Age: 42, Published: 1990-01-07] [Title: Java, Author: John Doe, Age: 50, Published: 1998-01-01] [Title: Python, Author: John Doe, Age: 62, Published: 2010-12-02] [Title: Scala, Author: John Doe, Age: 67, Published: 2015-03-02] [Title: HTML, Author: Mark Pickford, Age: 58, Published: 2008-07-02] [Title: Javascript, Author: James Pickford, Age: 52, Published: 2008-03-02] [Title: CSS, Author: Mark Keely, Age: 51, Published: 2008-04-02] [Title: ADA, Author: Ada Lovelace, Age: 44, Published: 1988-01-02] [Title: Dart, Author: Peter Dove, Age: 47, Published: 2020-01-02] [Title: Lisp, Author: William Fence, Age: 25, Published: 1977-09-24] [Title: Haskell, Author: William Fence, Age: 30, Published: 1982-04-14] [Title: C++, Author: William Fence, Age: 38, Published: 1990-04-02] [Title: C, Author: William Fence, Age: 38, Published: 1990-04-14] [Title: C#, Author: William Fence, Age: 60, Published: 2012-10-02]
ProcessByAuthor
Title: C++, Author: John Doe, Age: 42, Published: 1990-01-07 Title: C, Author: John Doe, Age: 42, Published: 1990-01-07 Title: Java, Author: John Doe, Age: 50, Published: 1998-01-01 Title: Python, Author: John Doe, Age: 62, Published: 2010-12-02 Title: Scala, Author: John Doe, Age: 67, Published: 2015-03-02 Title: HTML, Author: Mark Pickford, Age: 58, Published: 2008-07-02 Title: Javascript, Author: James Pickford, Age: 52, Published: 2008-03-02 Title: CSS, Author: Mark Keely, Age: 51, Published: 2008-04-02 Title: ADA, Author: Ada Lovelace, Age: 44, Published: 1988-01-02 Title: Dart, Author: Peter Dove, Age: 47, Published: 2020-01-02 Title: Lisp, Author: William Fence, Age: 25, Published: 1977-09-24 Title: Haskell, Author: William Fence, Age: 30, Published: 1982-04-14 Title: C++, Author: William Fence, Age: 38, Published: 1990-04-02 Title: C, Author: William Fence, Age: 38, Published: 1990-04-14 Title: C#, Author: William Fence, Age: 60, Published: 2012-10-02
ProcessByAuthorWithThreeOrMoreBooks
Title: C++, Author: John Doe, Age: 42, Published: 1990-01-07 Title: C, Author: John Doe, Age: 42, Published: 1990-01-07 Title: Java, Author: John Doe, Age: 50, Published: 1998-01-01 Title: Python, Author: John Doe, Age: 62, Published: 2010-12-02 Title: Scala, Author: John Doe, Age: 67, Published: 2015-03-02 Title: Lisp, Author: William Fence, Age: 25, Published: 1977-09-24 Title: Haskell, Author: William Fence, Age: 30, Published: 1982-04-14 Title: C++, Author: William Fence, Age: 38, Published: 1990-04-02 Title: C, Author: William Fence, Age: 38, Published: 1990-04-14 Title: C#, Author: William Fence, Age: 60, Published: 2012-10-02
ProcessByAuthorWithThreeOrMoreBooksIgnoreFirstTwo
Title: Java, Author: John Doe, Age: 50, Published: 1998-01-01 Title: Python, Author: John Doe, Age: 62, Published: 2010-12-02 Title: Scala, Author: John Doe, Age: 67, Published: 2015-03-02 Title: C++, Author: William Fence, Age: 38, Published: 1990-04-02 Title: C, Author: William Fence, Age: 38, Published: 1990-04-14 Title: C#, Author: William Fence, Age: 60, Published: 2012-10-02
ProcessByAuthorUsingHigherOrderFunction
Omit first two books where author has three or more books Title: Java, Author: John Doe, Age: 50, Published: 1998-01-01 Title: Python, Author: John Doe, Age: 62, Published: 2010-12-02 Title: Scala, Author: John Doe, Age: 67, Published: 2015-03-02 Title: C++, Author: William Fence, Age: 38, Published: 1990-04-02 Title: C, Author: William Fence, Age: 38, Published: 1990-04-14 Title: C#, Author: William Fence, Age: 60, Published: 2012-10-02
ProcessByAuthorUsingParameterisedFunction
Omit first two books where author has three or more books Title: Java, Author: John Doe, Age: 50, Published: 1998-01-01 Title: Python, Author: John Doe, Age: 62, Published: 2010-12-02 Title: Scala, Author: John Doe, Age: 67, Published: 2015-03-02 Title: C++, Author: William Fence, Age: 38, Published: 1990-04-02 Title: C, Author: William Fence, Age: 38, Published: 1990-04-14 Title: C#, Author: William Fence, Age: 60, Published: 2012-10-02 First book where author has three or more books Title: C++, Author: John Doe, Age: 42, Published: 1990-01-07 Title: Lisp, Author: William Fence, Age: 25, Published: 1977-09-24 Last book where author has three or more books Title: Scala, Author: John Doe, Age: 67, Published: 2015-03-02 Title: C#, Author: William Fence, Age: 60, Published: 2012-10-02
UniquePublishingDatesFromAuthorWithThreeOrMoreBooks
Unique signing events on day of publication by authors with three or more books. Date: 1977-09-24 Date: 1982-04-14 Date: 1990-01-07 Date: 1990-04-02 Date: 1990-04-14 Date: 1998-01-01 Date: 2010-12-02 Date: 2012-10-02 Date: 2015-03-02 Books From author with three or more books Title: C++, Author: John Doe, Age: 42, Published: 1990-01-07 Title: C, Author: John Doe, Age: 42, Published: 1990-01-07 Title: Java, Author: John Doe, Age: 50, Published: 1998-01-01 Title: Python, Author: John Doe, Age: 62, Published: 2010-12-02 Title: Scala, Author: John Doe, Age: 67, Published: 2015-03-02 Title: Lisp, Author: William Fence, Age: 25, Published: 1977-09-24 Title: Haskell, Author: William Fence, Age: 30, Published: 1982-04-14 Title: C++, Author: William Fence, Age: 38, Published: 1990-04-02 Title: C, Author: William Fence, Age: 38, Published: 1990-04-14 Title: C#, Author: William Fence, Age: 60, Published: 2012-10-02 There are 7 authors in total with any number of books
LibraryExample
John Doe William Fence Mark Keely James Pickford Mark Pickford
SplitBooksByEra
First book from each group, a new group starting at each modern book Title: Lisp, Author: William Fence, Age: 25, Published: 1977-09-24 Title: Javascript, Author: James Pickford, Age: 52, Published: 2008-03-02 Title: CSS, Author: Mark Keely, Age: 51, Published: 2008-04-02 Title: HTML, Author: Mark Pickford, Age: 58, Published: 2008-07-02 Title: Python, Author: John Doe, Age: 62, Published: 2010-12-02 Title: C#, Author: William Fence, Age: 60, Published: 2012-10-02 Title: Scala, Author: John Doe, Age: 67, Published: 2015-03-02 Title: Dart, Author: Peter Dove, Age: 47, Published: 2020-01-02
GroupBooksByEra
First book from each era (pre-2000 and 2000+) Title: Lisp, Author: William Fence, Age: 25, Published: 1977-09-24 Title: Javascript, Author: James Pickford, Age: 52, Published: 2008-03-02
ImperativeLibraryExample
John Doe Mark Pickford Mark Keely William Fence James Pickford