enumeration - Elixir - Looping through and adding to map

Question

I'm rebuilding something in Elixir from some code I built in C#.

It was pretty hacked together, but works perfectly (although not on Linux, hence rebuild).

Essentially what it did was check some RSS feeds and see if there was any new content. This is the code:

Map historic (URL as key, post title as value).
List<string> blogfeeds
while true
for each blog in blogfeeds
   List<RssPost> posts = getposts(blog)
   for each post in posts
        if post.url is not in historic
           dothing(post)
           historic.add(post)

I am wondering how I can do Enumeration effectively in Elixir. Also, it seems that my very process of adding things to "historic" is anti-functional programming.

Obviously the first step was declaring my list of URLs, but beyond that the enumeration idea is messing with my head. Could someone help me out? Thanks.

Answer 1

This is a nice challenge to have and solving it will definitely give you some insight into functional programming.

The solution for such problems in functional languages is usually reduce (often called fold). I will start with a short answer (and not a direct translation) but feel free to ask for a follow up.

The approach below will typically not work in functional programming languages:

map = %{}
Enum.each [1, 2, 3], fn x ->
  Map.put(map, x, x)
end
map

The map at the end will still be empty because we can't mutate data structures. Every time you call Map.put(map, x, x), it will return a new map. So we need to explicitly retrieve the new map after each enumeration.

We can achieve this in Elixir using reduce:

map = Enum.reduce [1, 2, 3], %{}, fn x, acc ->
  Map.put(acc, x, x)
end

Reduce will emit the result of the previous function as accumulator for the next item. After running the code above, the variable map will be %{1 => 1, 2 => 2, 3 => 3}.

For those reasons, we rarely use each on enumeration. Instead, we use the functions in the Enum module, that support a wide range of operations, eventually falling back to reduce when there is no other option.

EDIT: to answer the questions and go through a more direct translation of the code, this what you can do to check and update the map as you go:

Enum.reduce blogs, %{}, fn blog, history ->
  posts = get_posts(blog)
  Enum.reduce posts, history, fn post, history ->
    if Map.has_key?(history, post.url) do
      # Return the history unchanged
      history
    else
      do_thing(post)
      Map.put(history, post.url, true)
    end
  end
end

In fact, a set would be better here, so let's refactor this and use a set in the process:

def traverse_blogs(blogs) do
  Enum.reduce blogs, HashSet.new, &traverse_blog/2
end

def traverse_blog(blog, history) do
  Enum.reduce get_posts(blog), history, &traverse_post/2
end

def traverse_post(post, history) do
  if post.url in history do
    # Return the history unchanged
    history
  else
    do_thing(post)
    HashSet.put(history, post.url)
  end
end