Scala Snippets

This post is really just notes to myself, which I add to as I find things. But I thought I’d share — maybe others will find them useful.

0. Resources:

1. Use flatMap with monadish objects like Some/None to pick out (and map) the interesting stuff:

  val data  = Map( "name" -> "Fred", "age" -> 21, "iq" -> 140 )
  val attrs = List( "name", "age", "income" )
  attrs flatMap data.get   // List(Fred, 21); no iq or income, poor guy

2. Use partial functions to put cases in derived classes which get used from the base class. For example, here is how you would take care of handling bad input in one place (badValue) for a set of related classes:

  import scala.{ PartialFunction => PF }
  abstract class Base {
    def validCases: PF[...]  // define in subclasses
    def   badValue: PF[...] = { case value => err( "bad value: " + value ) }
    def handle( value:... ) = ( validCases orElse badValue )(value)
  class Derived1 extends Base {
    def validCases = {
      case ... => ...
      case ... => ...
  class Derived2 extends Base {
    def validCases = {
      case ... => ...
      case ... => ...

3. Use this.type in methods in a base class where you need to return the derived type (otherwise Scala will use the base type). Useful for builder hierarchies:

  abstract class Base {
    def withDefault(...):this.type = {
      this // but as the derived type!

4. If you are having type problems in Eclipse, try these in order:

  1. Save all files and wait a few moments.
  2. From the menu, do Project / Clean and wait a few moments.
  3. One by one, add types to closure args and method returns.
  4. Eliminate implicit defs, even if you are absolutely convinced that they aren’t coming into play in the problematic code.
  5. If you are getting a “missing parameter type” error where you think the compiler should be able to infer the type, try assigning an intermediate result to a variable to force the compiler to choose the most specific type.

5. From Daniel Sobral, an expression that, given a string containing words (and perhaps other crud), returns a map from words to their frequencies. \P{Alpha} is the negation of \p{Alpha}, the POSIX char class. See the JDK 6 Pattern javadoc for details on regex constructs. Note the use of identity, which is just ( x => x ); it’s defined in the prelude, so you don’t need to create another.

  val text = "How much wood could a wood chuck chuck, if a wood chuck could chuck wood."

6. Don’t forget that you can see the code scalac magically generates for you using a compiler option. The output below has been manually formatted (somewhat) and commented:

% cat Person.scala
 case class Person( name:String, age:Int )

% scalac -Xprint:typer Person.scala    # this is 2.9.1-final

[[syntax trees at end of typer]] // Scala source: Person.scala

package  {

// Here's the generated class.
case class Person extends java.lang.Object with ScalaObject with Product with Serializable {

  // Private members and public accessors.
  private[this] val name: String = _; def name: String =;
  private[this] val  age: Int    = _; def  age: Int    = Person.this.age;

  // Constructor.
  def this(name: String, age: Int): Person = {

  // Versions of copy() for every combination of supplied parameters.
  def copy(name: String = name, age: Int = age): Person = new Person(name, age)
  def copy$default$2: Int    @scala.annotation.unchecked.uncheckedVariance = Person.this.age
  def copy$default$1: String @scala.annotation.unchecked.uncheckedVariance =

  // The usual stuff you want for every immutable class.
  override def hashCode(): Int    = ScalaRunTime.this._hashCode(Person.this)
  override def toString(): String = ScalaRunTime.this._toString(Person.this)
  override def equals(x$1: Any): Boolean =
    Person.this.eq( x$1.asInstanceOf[java.lang.Object] ).||( x$1 match {
      case (name: String, age: Int)Person( (name$1 @ _), (age$1 @ _) ) if name$1.==(name).&&( age$1.==(age) )
            => x$1.asInstanceOf[Person].canEqual(Person.this)
     case _ => false
    } )
  override def canEqual(x$1: Any): Boolean = x$1.$isInstanceOf[Person]()

  // Methods that allow all case classes to be self-describing.
  override def productPrefix: java.lang.String = "Person";
  override def productArity: Int = 2;
  override def productElement(x$1: Int): Any = x$1 match {
    case 0 => name
    case 1 => age
    case _ => throw new java.lang.IndexOutOfBoundsException(x$1.toString())

}  // class Person

// Here's the companion object.
final object Person extends scala.runtime.AbstractFunction2[String,Int,Person]
  with ScalaObject with Serializable {

  def this(): object Person = {

  final override def toString(): java.lang.String = "Person";

  case def unapply(x$0: Person): Option[(String, Int)] =
    if (x$0.==(null))
        scala.Some.apply[(String, Int)](scala.Tuple2.apply[String, Int](x$, x$0.age));

  case def apply(name: String, age: Int): Person = new Person(name, age);

  protected def readResolve(): java.lang.Object = Person

}  // companion object

}  // package

7. The usual idiom for providing Java versions of Scala APIs:

  import scala.collection.JavaConverters._
  def jFoos = foos.asJava
  def  foos = ...

Info on Java-Scala conversions, and why it is better to avoid the implicit JavaConversions.  Info on JavaConversions.  Do this if you want exceptions checked in the Java code:

  @throws( classOf[TimeoutException] )
  def jFoos ...

8. Passing on repeated parameters (a.k.a. varargs), with modification:

  def foo( a:String, b:Int* ) = bar( a, 7 :: b.toList : _* )

And here’s how you match repeated parameters:

  case Seq( xs @ _* ) => ...

9. Don’t do this (unless you really mean to):

  val x = new X {...}

That leads to the use of reflection (x is structurally typed). This fixes it:

  val x:X = new X {...}

More details here.

10. To filter a Seq by type in a for comprehension, you need to use this trick:

val mixedTypes = List( 5, "five" )
  for (   (x:Int)y@(x:Int)

But if you use a pattern, it works without the trick:

val mixedTypes = List( (1,5), (1,"five") )
  for ( (a,b:Int)

Basically what’s happening is that if you use a pattern, you get a PartialFunction, which will handle values that match and ignore values that don’t. (x:Int) is not a pattern, but y@(x:Int) is.

11. If you are thinking in terms of zipWith or mapcar to combine multiple Seqs, this is probably what you want:

  val xs = List(1,2,3)
  val ys = List(7,8,9)
  (xs,ys).zipped map (_+_)  // List(8,10,12)

Note that no intermediate list is built, as is with

  xs zip ys map { case (x,y) => x+y }

and that zipped works for more than two input Seqs:

  (xs,ys,zs).zipped map ( _*_/_ )

Apparently this only works with Tuple2 and Tuple3, though. Also, be careful — zipped is strict, so attempting to map zipped infinite Streams will win you a StackOverflowError or OutOfMemoryError.

12. Here’s a discussion about profiling and performance. They rave about the YourKit profiler and the Caliper microbenchmarker, and point to this interesting post. This is all pre-macros, so the advice may need an update soon.

13. Here‘s how to use a PriorityQueue to get the top N (in this case 4) elements of a larger list by some ordering criterion, without having to sort the entire list:

  val pq = PriorityQueue( 7,3,4,1,2,0,9,5,8,6 )( Ordering.Int.reverse )
  Iterator.fill(4)( pq.dequeue ) foreach print  // 0123

Of course, a PriorityQueue is perfect for situations where you keep getting new entries and want to always process the highest-priority one first. BUT — currently there is no immutable PriorityQueue in the standard libary, only a mutable one. You might think that you could use SortedSet (TreeSet) as an immutable replacement for PriorityQueue, but it is the set’s Ordering that determines whether or not a value is already in the set, not equals() — so an attempt to add something to the collection which compares as equal to an existing member of the set will be ignored. For example, if you have a SortedSet of Vectors ordered on the size of the vector, then the set will only hold one vector of length 4. Only use SortedSet if you are sure that distinct values will never be considered equal by the set’s Ordering#compare.

14. GenTraversableFactory contains methods (which are available implicitly for Traversables) for creating/initializing new collections:

  collection.mutable.ArrayBuffer.fill(3,2)(1)           // 3x2 ArrayBuffer(s) of 1s
  List.fill(10)( util.Random.nextInt(2) )               // 10 of 0|1
  new String( Array.fill(20)( util.Random.nextPrintableChar ) )  // random String
  Vector.tabulate(3,3)( (i,j) => if (i==j) 1 else 0 )   // 3x3 identity matrix
  util.Random.shuffle( List.range(0,11,2) )             // 0,2,...,10 shuffled
  Iterator.fill(20)( util.Random.nextGaussian ) foreach println  // don't make a List
  List.iterate(1,10)(2*)                                // 1,2,4,8,...,512
  List.concat( 1 to 4, 9 to 12, Set(2,5) )              // 1,2,3,4,9,10,11,12,2,5

15. It’s not surprising that scala.util.Properties lets you get/set properties, but the same class will also tell you which OS you are on, your username, your temp dir, the values of environment variables, etc.

  Properties.propOrNone("foo") match { case None => ...; case Some(foo) => ... }
  Properties. envOrElse("foo","bar")

16. A nice post from Daniel Sobral (a Scala bard) explaining for comprehensions.

17. Tips for investigating long compile times.

18. Here’s the usual way of defining a Stream of Fibonacci numbers, using zip (remember that zipped won’t work because it’s strict):

  val fibs: Stream[BigInt] = 0 #:: 1 #:: ( fibs zip fibs.tail map ( n => n._1 + n._2 ) )

But you can make a more efficient version of this kind of Stream by avoiding the zip, like so:

  val fibs: Stream[BigInt] = {
    def loop( h:BigInt, n:BigInt ): Stream[BigInt] = h #:: loop( n, h+n )

19. To define cyclical dependencies:

  trait A { self:B => ... }
  trait B { self:A => ... }

20. The stackable trait pattern:

  abstract class       Ab { def f(...) : ... /* abstract */ } // or trait
  class Base   extends Ab { def f(...) = ... }
  trait Tweak1 extends Ab { abstract override def f(...) = super.f(...) }
  trait Tweak2 extends Ab { abstract override def f(...) = super.f(...) }

  val tweaked = new Base with Tweak1 with Tweak2    // or whatever
  tweaked.f(...)  // calls Tweak2.f calls Tweak1.f calls Base.f  // reverse order

If Ab is an abstract class (as opposed to a trait) it can take constructor parameters, but of course you can only extend one class.

21. Implicit value classes allow you to effectively add methods to a class without actually creating new objects:

  // This value class adds a times() method to Int, allowing you to do
  // 3 times { ... }
  implicit class Int_times(val numTimes:Int) extends AnyVal {
    def times(f: => _) {
      var i = numTimes
      while (i > 0) { f; i -= 1 }

22. To get the conveniences of case classes within a class hierarchy, make the case classes the leaves, duplicate any common fields in them (sorry DRY), and use abstract methods to represent the common fields in a superclass:

  // Want to be able to access node.value without having to figure out
  // whether it is a Branch or a Leaf.
  abstract class   Node { val value:Int }
  case     class Branch(      value:Int, left:Node, right:Node ) extends Node
  case     class   Leaf(      value:Int                        ) extends Node

23. Be careful with mapValues and filterKeys — they currently return a view (without saying so — a bug), which means that the computation gets done again every time you use the result.  Use map.mapValues(…).view.force to avoid this, or map the result through the identity function:

scala> val m = Map(1->2, 3->4) mapValues { x => println("OUCH!"); x+1 }
m: scala.collection.immutable.Map[Int,Int] = Map(1 -> 3, 3 -> 5)

scala> m(1)
res20: Int = 3

scala> val n = m map identity
n: scala.collection.immutable.Map[Int,Int] = Map(1 -> 3, 3 -> 5)

scala> n(1)
res21: Int = 3

24. Array equality is weird (a Java legacy, presumably):

  import collection.mutable
  import mutable.ArrayBuffer
          Set(1,2)        ==           Set(1,2)        // true
         List(1,2)        ==          List(1,2)        // true
  mutable.Set(1,2)        ==   mutable.Set(1,2)        // true
  ArrayBuffer(1,2)        ==   ArrayBuffer(1,2)        // true
        Array(1,2)        ==         Array(1,2)        // FALSE!!!
        Array(1,2)   sameElements    Array(1,2)        // true
        Array(1,2).deep   ==         Array(1,2).deep   // true
        Array(1,2).toSeq  ==         Array(1,2).toSeq  // true (WrappedArray)

25. Constructor goodies:

  class A         (                   b:Int )    // b is just a constructor arg
  class A         (               val b:Int )    // b is a field
  class A         (       private val b:Int )    // b is private
  class A private (                   b:Int )    // primary constructor is private,
                                                 // but usable by other constructors
  class A         ( @BeanProperty var b:Int )    // adds JavaBean getB/setB methods

And you can subclass a Java class with multiple constructors like so.

26. Composing Futures:

  import scala.concurrent._
  def f(x:Int) = { Thread.sleep(5000); x }
  val futures = 1 to 9 map { x => Future( f(x) ) }  // List[ Future[X] ]
  val future  = Future sequence futures             // Future[ List[X] ]
  future onSuccess { case results => println( results.sum ) }
  println("this gets printed before the sum")

27. To write a set of programs which all have to do the same setup (examine environment variables, process arguments, open connections, etc.) and teardown, rather than calling out to a library to do the common stuff you can simply extend App, put the common stuff in there, and then extend that to create each of your programs:

  class  DbApp  extends   App { /* common stuff */ }
  object DbApp1 extends DbApp { ... }
  object DbApp2 extends DbApp { ... }

The children of DbApp will of course inherit any variables defined by it.

28. You cannot use _root_ to import from the default package (ultimately a JVM limitation):

                class C1
  package a   { class C2 }
  package a.b { class C3 {
                  val c2fails = new C2            // not found (in a.b)
                  val c2works = new _root_.a.C2   // OK
                  val c1fails = new _root_.C1     // not a member of pkg 
              } }

29. To throw an exception without the overhead:

  throw new Exception with scala.util.control.NoStackTrace

30. You can use collection.breakout (the breakout method on the scala.collection package object) to provide a CanBuildFrom that produces a specific type, obviating a toList or other conversion.

  type S = String
  def cross( a:S, b:S ):List[S] = for ( c1 c2 => ""+c1+c2 ) )(collection.breakOut)  // ok
  cross("ab","12")  // List("a1","a2","b1","b2")

And here Rex Kerr shows how to use CanBuildFrom in your own code to make it return the type you started with.

31. While you can use “System exit 1” to exit a program just as you would in Java, “sys exit 1” is better in that it returns Nothing, allowing you to use it in expressions without losing the type info:

  def die( msg:String ) = { log(msg); sys exit 1 }
  val age = if ( born <= now ) now - born else die("born in the future!")

32. Do this to make an infinite Stream by repeating a TraversableOnce:

  implicit class RichTraversable[A](val items: TraversableOnce[A]) extends AnyVal {
    def looped =
      if (items.isEmpty)
        sys error ".looped"
      else {
        lazy val s: Stream[A] = items.toStream #::: s
List(1,2,3).looped take 10 foreach print  // 1231231231

But that’s still a Stream, so it will eventually consume the heap storing calculated results. You could of course make an Iterator, e.g.

  implicit class TraversableLooper[A : scala.reflect.ClassTag](val items: TraversableOnce[A]) {
    def looped: Iterator[A] =
      if (items.isEmpty)
        sys error "(empty).looped"
        new Iterator[A] {
          val array = items.toArray
          val numItems = array.length
          var i = -1
          def hasNext = true
          def next = {
            i += 1
            if (i == numItems) i = 0
  (List(1,2,3).looped take 900000000).sum   // 1800000000

Note that you can’t currently make the latter a value class because it defines an inner class. That restriction is probably going to be eliminated (in 2.11?).  And notice the use of ClassTag so that we can create an Array[A] without knowing what A is.

33. You can do this to choose between methods once (and then use whichever you have chosen repeatedly):

  val f = if (...) f1 _ else f2 _

But if f1 and/or f2 are overloaded you will probably need to say which you mean, like so:

  val f = if (...) f1(_:Int) else f2(_:Int)

More info here.

34. Reverse sorting:

  val names = List("d","b","c","a")
  println( names sorted Ordering[String].reverse )

or, more generally:

  def revSort[ A:Ordering ]( s:Seq[A] ) = s sorted Ordering[A].reverse

Or you could (apparently) add a method to Seq:

  implicit class SeqMethods[A](val xs: Seq[A]) extends AnyVal {
    def revSorted(implicit ord: Ordering[A]) = xs sorted ord.reverse

35. You can bind a lowercase variable to an existential type:

  val x: Some[_] = Some("foo")

  x match {
    case st:Some[t] => // t is a type variable
      val it:t = st.get
      var items = Set[t]() // which we can use to define new types
      items += it
  } // Set[_]

Note that you still get Set[_] even if you start with Some[String], because Some[t] would match other types; this doesn’t bind t to String.

36. You can exclude some parameters of a case class from the usual magic by putting them in a separate parameter list.  E.g. here see that c is ignored by toString() and equals():

scala> case class Foo(a: Int, b: Int)(c: Int)
defined class Foo
scala> val f1 = Foo(1,2)(3)
f1: Foo = Foo(1,2)
scala> val f2 = Foo(4,5)(6)
f2: Foo = Foo(4,5)
scala> val f3 = Foo(1,2)(7)
f3: Foo = Foo(1,2)
scala> Set(f1,f2,f3)
res1: scala.collection.immutable.Set[Foo] = Set(Foo(1,2), Foo(4,5))

[other Scala posts]


  1. Posted January 10, 2013 at 1:53 pm | Permalink | Reply

    This is the most interesting yet so simple example of flatMap that I’ve seen. Thank you for these note!

  2. Posted May 19, 2014 at 11:54 pm | Permalink | Reply

    I enjoyed this deterministic and scientific approach to eclipse:

    4. If you are having type problems in Eclipse, try these in order:

    Save all files and wait a few moments.
    From the menu, do Project / Clean and wait a few moments.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

%d bloggers like this: