This post is really just notes to myself, which I intend to add to as I go. But I thought I’d share — maybe others will find them useful.
0. Resources:
- Here’s the latest source for Scala.
- Here is the main docs page, the current API, the Intro to Collections, and its sections on views and performance characteristics. Seq traits. Language reference.
- Scala Roadmap (discussion) and Issues. Akka Roadmap.
- Typesafe blog. New in 2.10 (slides, type tags).
- StackOverflow’s Scala page. Some lesser-known features of Scala.
- Planet Scala, LangRef, Daily Scala, Scala Puzzlers, Literate Programming, 99 Scala Solutions, Twitter’s Scala School (advanced types) and Effective Scala,
- Online books (somewhat pre-2.8): Programming in Scala 1st edition (object equality), Programming Scala (type system, higher-kinded types).
- Coursera course.
- Nice post on error handling. Initialization order. Variance (compared to Java, a bit out of date). Typeclasses. Another tour.
- Scala Community.
1. Use flatMap with monadish objects like Some/None to pick out (and map) the interesting stuff:
val data = Map( "name" -> "Fred", "age" -> 21, "iq" -> 140 ) val attrs = List( "name", "age", "income" ) attrs flatMap ( data get ) // List(Fred, 21); no iq or income, poor guy
2. Use partial functions to put cases in derived classes which get used from the base class. For example, here is how you would take care of handling bad input in one place (badValue) for a set of related classes:
import scala.{ PartialFunction => PF }
abstract class Base {
def validCases: PF[...] // define in subclasses
def badValue: PF[...] = { case value => err( "bad value: " + value ) }
def handle( value:... ) = ( validCases orElse badValue )(value)
}
class Derived1 extends Base {
def validCases = {
case ... => ...
case ... => ...
}
class Derived2 extends Base {
def validCases = {
case ... => ...
case ... => ...
}
}
3. Use this.type in methods in a base class where you need to return the derived type (otherwise Scala will use the base type). Useful for builder hierarchies:
abstract class Base {
def withDefault(...):this.type = {
...
this // but as the derived type!
}
}
4. If you are having type problems in Eclipse, try these in order:
- Save all files and wait a few moments.
- From the menu, do Project / Clean and wait a few moments.
- One by one, add types to closure args and method returns.
- Eliminate implicit defs, even if you are absolutely convinced that they aren’t coming into play in the problematic code.
- If you are getting a “missing parameter type” error where you think the compiler should be able to infer the type, try assigning an intermediate result to a variable to force the compiler to choose the most specific type.
5. From Daniel Sobral, an expression that, given a string containing words (and perhaps other crud), returns a map from words to their frequencies. \P{Alpha} is the negation of \p{Alpha}, the POSIX char class. See the JDK 6 Pattern javadoc for details on regex constructs. Note the use of identity, which is just ( x => x ); it’s defined in the prelude, so you don’t need to create another.
val text = "How much wood could a wood chuck chuck, if a wood chuck could chuck wood."
text.toLowerCase.split("""\P{Alpha}+""").groupBy(identity).mapValues(_.length)
6. Don’t forget that you can see the code scalac magically generates for you using a compiler option. The output below has been manually formatted (somewhat) and commented:
% cat Person.scala
case class Person( name:String, age:Int )
% scalac -Xprint:typer Person.scala # this is 2.9.1-final
[[syntax trees at end of typer]] // Scala source: Person.scala
package {
// Here's the generated class.
case class Person extends java.lang.Object with ScalaObject with Product with Serializable {
// Private members and public accessors.
private[this] val name: String = _; def name: String = Person.this.name;
private[this] val age: Int = _; def age: Int = Person.this.age;
// Constructor.
def this(name: String, age: Int): Person = {
Person.super.this();
()
}
// Versions of copy() for every combination of supplied parameters.
def copy(name: String = name, age: Int = age): Person = new Person(name, age)
def copy$default$2: Int @scala.annotation.unchecked.uncheckedVariance = Person.this.age
def copy$default$1: String @scala.annotation.unchecked.uncheckedVariance = Person.this.name
// The usual stuff you want for every immutable class.
override def hashCode(): Int = ScalaRunTime.this._hashCode(Person.this)
override def toString(): String = ScalaRunTime.this._toString(Person.this)
override def equals(x$1: Any): Boolean =
Person.this.eq( x$1.asInstanceOf[java.lang.Object] ).||( x$1 match {
case (name: String, age: Int)Person( (name$1 @ _), (age$1 @ _) ) if name$1.==(name).&&( age$1.==(age) )
=> x$1.asInstanceOf[Person].canEqual(Person.this)
case _ => false
} )
override def canEqual(x$1: Any): Boolean = x$1.$isInstanceOf[Person]()
// Methods that allow all case classes to be self-describing.
override def productPrefix: java.lang.String = "Person";
override def productArity: Int = 2;
override def productElement(x$1: Int): Any = x$1 match {
case 0 => name
case 1 => age
case _ => throw new java.lang.IndexOutOfBoundsException(x$1.toString())
}
} // class Person
// Here's the companion object.
final object Person extends scala.runtime.AbstractFunction2[String,Int,Person]
with ScalaObject with Serializable {
def this(): object Person = {
Person.super.this();
()
};
final override def toString(): java.lang.String = "Person";
case def unapply(x$0: Person): Option[(String, Int)] =
if (x$0.==(null))
scala.this.None
else
scala.Some.apply[(String, Int)](scala.Tuple2.apply[String, Int](x$0.name, x$0.age));
case def apply(name: String, age: Int): Person = new Person(name, age);
protected def readResolve(): java.lang.Object = Person
} // companion object
} // package
7. The usual idiom for providing Java versions of Scala APIs:
import scala.collection.JavaConverters._ def jFoos = foos.asJava def foos = ...
Info on Java-Scala conversions, and why it is better to avoid the implicit JavaConversions. Info on JavaConversions. Do this if you want exceptions checked in the Java code:
@throws( classOf[TimeoutException] ) def jFoos ...
8. Passing on repeated parameters (a.k.a. varargs), with modification:
def foo( a:String, b:Int* ) = bar( a, 7 :: b.toList : _* )
And here’s how you match repeated parameters:
case Seq( xs @ _* ) => ...
9. Don’t do this (unless you really mean to):
val x = new X {...}
That leads to the use of reflection (x is structurally typed). This fixes it:
val x:X = new X {...}
More details here.
10. To filter a Seq by type in a for comprehension, you need to use this trick:
val mixedTypes = List( 5, "five" ) for ( (x:Int) <- mixedTypes ) println(x) // type mismatch for ( y@(x:Int) <- mixedTypes ) println(x) // 5
But somehow it works for tuples without the trick:
val mixedTypes = List( (1,5), (1,"five") ) for ( (a,b:Int) <- mixedTypes ) println(b) // 5
11. If you are thinking in terms of zipWith or mapcar to combine multiple Seqs, this is probably what you want:
val xs = List(1,2,3) val ys = List(7,8,9) (xs,ys).zipped map (_+_) // List(8,10,12)
Note that no intermediate list is built, as is with
xs zip ys map { case (x,y) => x+y }
and that zipped works for more than two input Seqs:
(xs,ys,zs).zipped map ( _*_/_ )
Apparently this only works with Tuple2 and Tuple3, though.
Also, be careful – zipped is strict, so attempting to map zipped infinite Streams will win you a StackOverflowError or OutOfMemoryError.
12. Here’s a discussion about profiling and performance. They rave about the YourKit profiler and the Caliper microbenchmarker, and point to this interesting post. This is all pre-macros, so the advice may need an update soon.
13. Here‘s how to use a PriorityQueue to get the top N (in this case 4) elements of a larger list by some ordering criterion, without having to sort the entire list:
val pq = PriorityQueue( 7,3,4,1,2,0,9,5,8,6 )( Ordering.Int.reverse ) Iterator.fill(4)( pq.dequeue ) foreach print // 0123
Of course, a PriorityQueue is perfect for situations where you keep getting new entries and want to always process the highest-priority one first.
14. GenTraversableFactory contains methods (which are available implicitly for Traversables) for creating/initializing new collections:
collection.mutable.ArrayBuffer.fill(3,2)(1) // 3x2 ArrayBuffer(s) of 1s
List.fill(10)( util.Random.nextInt(2) ) // 10 of 0|1
new String( Array.fill(20)( util.Random.nextPrintableChar ) ) // random String
Vector.tabulate(3,3)( (i,j) => if (i==j) 1 else 0 ) // 3x3 identity matrix
util.Random.shuffle( List.range(0,11,2) ) // 0,2,...,10 shuffled
Iterator.fill(20)( util.Random.nextGaussian ) foreach println // don't make a List
List.iterate(1,10)(2*) // 1,2,4,8,...,512
List.concat( 1 to 4, 9 to 12, Set(2,5) ) // 1,2,3,4,9,10,11,12,2,5
15. It’s not surprising that scala.util.Properties lets you get/set properties, but the same class will also tell you which OS you are on, your username, your temp dir, the values of environment variables, etc.
Properties.propOrNone("foo") match { case None => ...; case Some(foo) => ... }
Properties. envOrElse("foo","bar")
16. A nice post from Daniel Sobral (a Scala bard) explaining for comprehensions.
17. Tips for investigating long compile times. 2.10 is supposed to improve pattern-matching performance.
18. Here’s the usual way of defining a Stream of Fibonacci numbers, using zip (remember that zipped won’t work because it’s strict):
val fibs: Stream[BigInt] = 0 #:: 1 #:: ( fibs zip fibs.tail map ( n => n._1 + n._2 ) )
But you can make a more efficient version of this kind of Stream by avoiding the zip, like so:
val fibs: Stream[BigInt] = {
def loop( h:BigInt, n:BigInt ): Stream[BigInt] = h #:: loop( n, h+n )
loop(0,1)
}
19. To define cyclical dependencies:
trait A { self:B => ... }
trait B { self:A => ... }20. The stackable trait pattern:
abstract class Ab { def f(...) : ... /* abstract */ } // or trait
class Base extends Ab { def f(...) = ... }
trait Tweak1 extends Ab { abstract override def f(...) = super.f(...) }
trait Tweak2 extends Ab { abstract override def f(...) = super.f(...) }
val tweaked = new Base with Tweak1 with Tweak2 // or whatever
tweaked.f(...) // calls Tweak2.f calls Tweak1.f calls Base.f // reverse order
If Ab is an abstract class (as opposed to a trait) it can take constructor parameters, but of course you can only extend one class.
21. Implicit value classes allow you to effectively add methods to a class without actually creating new objects:
// This value class adds a times() method to Int, allowing you to do
// 3 times { ... }
implicit class Int_times( val numTimes:Int ) extends AnyVal {
def times( f: => _ ) {
var i = numTimes
while ( i>0 ) { f; i -= 1 }
}
}
22. To get the conveniences of case classes within a class hierarchy, make the case classes the leaves, duplicate any common fields in them (sorry DRY), and use abstract methods to represent them in a superclass:
// Want to be able to access node.value without having to figure out
// whether it is a Branch or a Leaf.
abstract class Node { def value:Int }
case class Branch( value:Int, left:Node, right:Node ) extends Node
case class Leaf( value:Int ) extends Node
23. Be careful with mapValues and filterKeys — they currently return a view (without saying so — a bug), which means that the computation gets done again every time you use the result. Use map.mapValues(…).view.force to avoid this. I hope they change this so that we don’t get a view unless we ask for it.
24. Array equality is weird (a Java legacy, presumably):
import collection.mutable
Set(1,2) == Set(1,2) // true
List(1,2) == List(1,2) // true
mutable. Set(1,2) == mutable. Set(1,2) // true
mutable.ArrayBuffer(1,2) == mutable.ArrayBuffer(1,2) // true
Array(1,2) == Array(1,2) // FALSE!!!
Array(1,2) sameElements Array(1,2) // true
25. Constructor goodies:
class A ( b:Int ) // b is just a constructor arg
class A ( val b:Int ) // b is a field
class A ( private val b:Int ) // b is private
class A private ( b:Int ) // primary constructor is private,
// but usable by other constructors
class A ( @BeanProperty var b:Int ) // adds JavaBean getB/setB methods
And you can subclass a Java class with multiple constructors like so.
import scala.concurrent._
import ExecutionContext.Implicits.global
def f(x:Int) = { Thread.sleep(5000); x }
val futures = 1 to 9 map { x => Future( f(x) ) } // List[ Future[X] ]
val future = Future.sequence(futures) // Future[ List[X] ]
future onSuccess { case results => println( results.sum ) }
println("this gets printed before the sum")
27. To write a set of programs which all have to do the same setup (examine environment variables, process arguments, open connections, etc.) and teardown, rather than calling out to a library to do the common stuff you can simply extend App, put the common stuff in there, and then extend that to create each of your programs:
class DbApp extends App { /* common stuff */ }
object DbApp1 extends DbApp { ... }
object DbApp2 extends DbApp { ... }
The children of DbApp will of course inherit any variables defined by it.
28. You cannot use _root_ to import from the default package (ultimately a JVM limitation):
class C1
package a { class C2 }
package a.b { class C3 {
val c2fails = new C2 // not found (in a.b)
val c2works = new _root_.a.C2 // OK
val c1fails = new _root_.C1 // not a member of pkg <root>
} }
29. To throw an exception without the overhead:
throw new Exception with scala.util.control.NoStackTrace
30. You can use collection.breakout (the breakout method on the scala.collection package object) to provide a CanBuildFrom that produces a specific type, obviating a toList or other conversion.
type S = String
def cross( a:S, b:S ):List[S] = for ( c1 <- a; c2 <- b ) yield ""+c1+c2 // type mismatch (Vector)
def cross( a:S, b:S ):List[S] = a.flatMap( c1 => b.map( c2 => ""+c1+c2 ) )(collection.breakOut) // ok
cross("ab","12") // List("a1","a2","b1","b2")
And here Rex Kerr shows how to use CanBuildFrom in your own code to make it return the type you started with.
31. While you can use “System exit 1″ to exit a program just as you would in Java, “sys exit 1″ is better in that it returns Nothing, allowing you to use it in expressions without losing the type info:
def die( msg:String ) = { log(msg); sys exit 1 }
val age = if ( born <= now ) now - born else die("born in the future!")
32. Do this to make an infinite Stream by repeating a finite Stream (without taking up infinite memory or doing more allocations than necessary):
implicit class LoopedStream[T]( s:Stream[T] ) {
lazy val looped: Stream[T] = s #::: looped
}
Stream(1,2,3).looped take 10 foreach print // 1231231231
2 Comments
This is the most interesting yet so simple example of flatMap that I’ve seen. Thank you for these note!
Glad you found them helpful!