Anti-Phishing Measures in Tellody

(Cross-posting of https://www.tellody.com/blog/166-anti-phishing-measures-in-tellody)

Sending bulk emails is a risky business because, on the one hand, for Tellody that is running it, it’s prone to be abused by scammers and, on the other, for our users, it is easy to create emails in good faith but inadvertently contain indicators that will taint them.

Protecting ourselves and our users is an open-ended issue without easy solutions, but we decided that something is better than nothing and some action is better than no action. We tried to implement some kind of spoofing detection, as a first step. In this article, I’ll briefly explain what we have implemented, what you have to look out for and how you can fix your email if it gets flagged. Since we target phishers, the component is called “CoastGuard”.

I will use the following sample screen that contains all current notifications, as my reference.

phishing-indicators-2

 

First of all, you should not mix scripts in your words. Mixing scripts is a prime indicator of spoofing because words can appear as meaning one thing without actually writing them the way they are supposed to be. For example, the “Spoofed” notification appears to refer to the word “Paypal” but it mixes two scripts, “Basic Latin” (what “Paypal” is supposed to be written in) and “Greek and Coptic”. In this case it’s the uppercase Greek Rho, indistinguishable in all manners from the Latin uppercase P, that triggers this notification. There’s no legitimate reason to use more than one script within a word, and there are plenty of illegitimate ones so, if you find yourself needing to use a particularly clever brand name, that you just thought of, which uses more than one script, Tellody will not allow you to send it in an email. Get in touch if this is something essential to you.

Second, Punycodes have been shown to present the same spoofing opportunities, with little added value. The Punycode that is rejected above looks the same as https://www.epic.com but is not the same URL. Tellody rejects punycodes altogether, at least until major browsers start presenting adequate warnings about them. Again, if using a punycode is essential to you, get in touch with us.

The third category of spoofing indicator is when we think that someone tries to masquerade link text to make it appear as if it’s a URL. Maybe not many people would be fooled by “http:\\”, but would you be fooled by “http:⁄⁄”, containing a character that looks a lot like a forward slash? I know I would.

The next two indicators pertain to using the plethora of Unicode characters to make a string look as if it’s contiguous but is actually broken by characters with no width or scarcely any width. Adding such a pseudo-space character can create a string that looks to the eye like “Viagra” but fails to match a filter that looks for an exact match.

Finally, the last indicator is shown when the text of a link is something that looks as if it is a URL but it is different than the actual link target. A URL that appears to go to http://www.apple.com, while it goes to some shady site, is rejected by Tellody. Note that “appears” means how we think the text appears to a human and, the more tricks we detect to make something look like a URL while failing the straightforward syntactic check for one, the worse off we deem it to be.

This is a first version of those checks and they might evolve over time. If you find yourself having a legitimate business need that is hampered, please get in touch with info@tellody.com to discuss it.

Advertisements

Let’s get physical

I’m a mechanical engineer and, although I studied for software engineering degrees right after I graduated, I did my fair share of numerical calculations during my five years of studies in engineering. Some of it in Turbo Pascal, but most of it in the recommended language, which was Fortran. It was very spartan but, one thing it did well, was get out of the way between engineers and their calculations. Around 2007, 17 full years away from purely numerical computing, I turned around to see what had changed in the domain of calculating with physical quantities. Believe me, any new version of Fortran that has come out was a huge improvement over Fortran 77, but I found that not much had changed in the calculations themselves. In particular, regarding calculations with physical units and proper dimensional analysis. The kind that could avoid disasters like these.

Before I tell you what I concocted in 2007 and then in 2014, let me stay a little longer on Fortran, or rather what was designed to be a modern-day Fortran: Fortress.

Fortress

Fortress, the brainchild of Guy Steele, was an experimental project in Sun to answer the HPCS challenge (cf. “Fwd: HPCS recap”). After it was not selected in subsequent rounds of the challenge, the project persisted for a total of ten years, before being axed by Oracle. People explain things like that because of the “pragmatic” approach of Oracle towards Java. Which is a polite way to say that Oracle makes Java walk the streets for money, and probably gives her a good beatin’ whenever she comes back with low earnings.

Fortress had nailed the dimension thing, offering dimensions (decribed as “type-like constructs that are separate from other types”) and units (described as “special constructs that modify types and values […] that are instances of dimensions”). Dimensionless and dimensioned numbers can use the whole gamut of language constructs. In addition to standard SI dimensions and units, many more are available in the standard library and more can be added.

Here is what some dimension-aware calculations would look in Fortress (Fortress is a language written to be read back as close to mathematics as possible, so this is how Fortress code would look like).

x:ℝ64 Length = 1.3m
t:ℝ64 Time = 5 s
v:ℝ64 Velocity = x/t
w:ℝ64 Velocity in nm/s = 17nm/s

Standard SI prefixes like (nano, micro etc) and their abbreviations (n, μ etc) can be used. In order to avoid misunderstandings, these prefixes cannot be used in front of units of dimension Information (bit and byte).

Both dimensions and units can be multiplied, divided, and raised to rational powers to produce new dimensions and units. Multiplication can be expressed using juxtaposition or · (yes, Fortress actually gives the natural meaning of multiplication to juxtaposition!) Division can be expressed using / or per. The syntactic operators square and cubic (or the special postfix syntactic operators squared and cubed) may be applied to a dimension or unit in order
to raise it to the second power, third power, respectively. The syntactic operator inverse may be applied to a dimension or unit to divide it into 1. Ex. grams per cubic centimeter, meter per second squared, inverse ohms.

All in all, a pretty solid treatment. Unfortunately, none of this was actually implemented… However, there is one existing language which does support dimensioned calculations in a very real way.

F#

F# is the only mainstream language, as far as I know, that contains specific constructs for working with units (Curl supposedly does also, but I’m not familiar with it and it’s a long stretch to call it mainstream).

It is not entirely pleasing as, there is no concept of dimension, only the concept of unit of measure, but all the machinery needed for safe dimensional calculations is there. As is common in the .Net and Java world, the units of measure are defined using an attribute (annotation in Java-speak). SI predefined “measures” can be found in the PowerPack.

[<Measure>] type kg
[<Measure>] type s
[<Measure>] type m
[<Measure>] type cm = m / 100
[<Measure>] type ml = cm^3

let gravityOnEarth = 9.81<m/s^2>
let heightOfFall = 15<m>
let speedOfImpact = sqrt(2.0 * gravityOnEarth * heightOfFall)
let someForce = 10<kg m s^-2>

As you see in the example, units can be defined by their own (in which case they can be thought as defining a dimension) or as derived from some unit expression. Such expressions can be suffixed to numeric literals between angle brackets (without any intervening space). As a nod to Fortress, who knows?, juxtaposition stands for multiplication in the numerator of unit expressions. A dimensionless quantity is denoted by the unit expression <1>. One can even have generic units! And, you know, .Net has real multidimensional array types, and value types and also arrays of value types. The whole makes a compelling proposal for a scientist or an engineer.

Dimenso
Back in 2007, none of that was there. So I set myself a goal of experimenting in that domain, leading to the open-source project Dimenso. I focused on just SI quantities and then, just the MKS part together with current (no temperature, no amount of substance and no luminous intensity).

Dimenso: C++

The prototypical implementation was in C++. Believe it or not, the template language in C++ is pretty advanced, even with today’s standards. One could really make a case that C++ template metaprogramming is akin to using dependent types. Because templates in C++, unlike generics in the likes of Java or .Net, can actually be parameterized by integers, in addition to types. Here’s how the definition of the class that embodies dimensioned values would start: template<int L, int M, int T, int C> class DimensionedValue.

Defining what it means to multiply two dimensioned values together is this simple operator (yes! Java-lovers, most other languages do have operator overloads!).

template<int L2, int M2, int T2, int C2>
 DimensionedValue<L+L2,M+M2,T+T2,C+C2> operator*(DimensionedValue<L2,M2,T2,C2> other) {
     return DimensionedValue<L+L2,M+M2,T+T2,C+C2>(magnitude()*other.magnitude());
}

Given all the machinery available in C++, one can really make these objects look and behave like plain numbers. The challenge, then, was to go as close to that in C# and Java, which lacked support for dependent types.

Dimenso: C# and Java

The way I followed was: code generation. I generated all dimension combinations within specific bounds (but I also provided the Python script if one would need a more extended set). These combinations were named after the corresponding exponents (for example, type m_2k0s1a0 represents dimension L-2T) but actual names from physics were used, when available (but not in a right way… if you try to use Dimenso, stick to the ugly names).

This is how an arithmetic operation on “length” would look like in Java, if I hadn’t messed up the name thing.

public velocity divide(time v2) {
    velocity ret = new velocity(this.value/v2.getValue());
    return ret;
}

In C#, which does have operator overloads and value types, the whole is a little more palatable. But still, generating hundreds of classes feels… kind of wrong. Let’s see what the Java of today has to say on the matter.

JSR-275, JSR-363 and JScience
To make a long story short, JSR-275 “Measures and Units” was finally rejected and, its successor, JSR-363 “Units of Measurement API” has not really formed yet but, abandoning at that point would make it really short. That’s why I’m going to look at support for units in JScience, which claims support for such.

Let’s look at some sample code, which will highlight the point I was trying to make with Dimenso.

// Compiler Warnings
Unit<Acceleration> mps2 = Unit.valueOf("m/s²"); // Compiler Warning.
Unit<Power> WATT = JOULE.divide(SECOND); // Compiler Warning.

// Adding run-time check of dimension consistency removes the warning.
Unit<Acceleration> mps2 = Unit.valueOf("m/s²")
    .asType(Acceleration.class); 
Unit<Power> WATT = JOULE.divide(SECOND).asType(Power.class);

// Use wildcard parameterized type (no warning and no check).
Unit<?> kelvinPerSec = KELVIN.divide(SECOND);

JScience has a lot of capabilities for units, including arbitrary systems of dimensions, localizable unit parsing and output and compound units, like those for common time. But there is no way to have the compiler figure out what units two quantities multiplied or divided among themselves have. This is a shortcoming of Java, not of JScience. And, unless you’ve been ruined by too many years of Java, you can tell that no scientist or engineer would put up with so much unwarranted verbosity to do his job.

Do not despair, as Java is not the only choice you have on the JVM.

Scala

Let’s see what I’ve been able to do with some typelevel hacking in Scala. First of all, I’d like to point you to a better solution, Metascala Units, which did come up in my Google search, and a tab in my browser was open, waiting patiently for me to have a look at them, which I did, but after the fact. No matter, because what I really wanted was some practical experience with typelevel programming, with dimension-aware calculations as a good case to work on. The way that Metascala encodes integers in the typesystem is a lot more concise and a lot more elegant than my own, which is based on Shapeless’ encoding of naturals.

Here is some safe code using Metascala Units:

val dist : Length = m(2.3)
val time : Time = s(1.7)
val x = dist * time
val speed : Speed = dist / time

I hope you can appreciate the fact that this looks like using built-in features. It is as concise as it can get and offers full compile-time checking.

And here’s my own stab at this. It’s not part of Dimenso yet, because it’s not fully tested, I believe that if I build stronger typelevel kung-fu, I’ll be able to handle output of units at compile-time, as well, and there is probably some more magic to add with implicit value classes.

case class DimVal[M1 <: ZInt, K1 <: ZInt, S1 <: ZInt, A1 <: ZInt](val value : Double) {
    type T = DimVal[M1,K1,S1,A1]
    def +(b: DimVal[M1,K1,S1,A1]) = new DimVal[M1,K1,S1,A1](value + b.value)

    def -(b: DimVal[M1,K1,S1,A1]) = new DimVal[M1,K1,S1,A1](value - b.value)

    def *[M2 <: ZInt, M <: ZInt, K2 <: ZInt, K <: ZInt, S2 <: ZInt, S <: ZInt, A2 <: ZInt, A <: ZInt]
    (b: DimVal[M2,K2,S2,A2])
    (implicit ev: ZSumAux[M1,M2,M], ev2: ZSumAux[K1,K2,K], ev3: ZSumAux[S1,S2,S], ev4: ZSumAux[A1,A2,A]) : DimVal[M,K,S,A] =
        new DimVal[M,K,S,A](value * b.value)

    def /[M2 <: ZInt, M <: ZInt, K2 <: ZInt, K <: ZInt, S2 <: ZInt, S <: ZInt, A2 <: ZInt, A <: ZInt]
    (b: DimVal[M2,K2,S2,A2])
    (implicit ev: ZDiffAux[M1,M2,M], ev2: ZDiffAux[K1,K2,K], ev3: ZDiffAux[S1,S2,S], ev4: ZDiffAux[A1,A2,A]) : DimVal[M,K,S,A] =
        new DimVal[M,K,S,A](value / b.value)

    def asString () (implicit i1: ToZInt[M1], i2: ToZInt[K1], i3: ToZInt[S1], i4: ToZInt[A1]) = {
        val x1 = i1.apply()
        val x2 = i2.apply()
        val x3 = i3.apply()
        val x4 = i4.apply()
        val u =
        (x1,x2,x3,x4) match {
            case (0,0,1,1) => " C"
            case (1,1,-2,0) => " N"
// ... more special cases to handle derived units elided ...
            case _ => {
                val sb = new StringBuilder
// ... ugly, imperative code elided ...
                if(sb.length > 0) " "+sb else ""
            }
        }
        value.toString + u
    }

}

The dimension calculations are done at the type level, using types ZSumAux and ZDiffAux, and type ToZInt handles conversion to language-level integers. Like I said, not all induction cases are tested, but here’s what they look like now.

import shapeless._
import shapeless.Nat._0

trait ZInt

case class ZPos[T <: Nat]() extends ZInt
case class ZNeg[T <: Nat]() extends ZInt

trait ZSumAux[A <: ZInt, B <: ZInt, C <: ZInt]
trait ZDiffAux[A <: ZInt, B <: ZInt, C <: ZInt]

object ZSumAux {
    implicit def sum3[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : SumAux[A, B, C]) = new ZSumAux[ZPos[A], ZPos[B], ZPos[C]] {}

    implicit def sum4[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : SumAux[A, B, C]) = new ZSumAux[ZNeg[A], ZNeg[B], ZNeg[C]] {}

    implicit def sum5[A <: Nat] = new ZSumAux[ZPos[A], ZNeg[A], ZPos[_0]] {}

    implicit def sum6[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : DiffAux[A, B, C], ev2: LT[B,A]) = new ZSumAux[ZPos[A], ZNeg[B], ZPos[C]] {}

    implicit def sum7[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : DiffAux[B, A, C], ev2: LT[A,B]) = new ZSumAux[ZPos[A], ZNeg[B], ZNeg[C]] {}

    implicit def sum8[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : DiffAux[A, B, C], ev2: LT[B,A]) = new ZSumAux[ZNeg[A], ZPos[B], ZNeg[C]] {}

    implicit def sum9[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : DiffAux[B, A, C], ev2: LT[A,B]) = new ZSumAux[ZNeg[A], ZPos[B], ZPos[C]] {}
}

object ZDiffAux {
    implicit def diff3[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : DiffAux[A, B, C], ev2: LT[B,A]) = new ZDiffAux[ZPos[A], ZPos[B], ZPos[C]] {}

    implicit def diff3b[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : DiffAux[B, A, C], ev2: LT[A,B]) = new ZDiffAux[ZPos[A], ZPos[B], ZNeg[C]] {}

    implicit def diff4[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : DiffAux[A, B, C], ev2: LT[B,A]) = new ZDiffAux[ZNeg[A], ZNeg[B], ZNeg[C]] {}

    implicit def diff4b[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : DiffAux[B, A, C], ev2: LT[A,B]) = new ZDiffAux[ZNeg[A], ZNeg[B], ZPos[C]] {}

    implicit def diff5[A <: Nat] = new ZDiffAux[ZPos[A], ZPos[A], ZPos[_0]] {}

    implicit def diff5b[A <: Nat] = new ZDiffAux[ZNeg[A], ZNeg[A], ZPos[_0]] {}

    implicit def diff6[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : SumAux[A, B, C], ev2: LT[B,A]) = new ZDiffAux[ZPos[A], ZNeg[B], ZPos[C]] {}

    implicit def diff7[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : SumAux[B, A, C], ev2: LT[A,B]) = new ZDiffAux[ZPos[A], ZNeg[B], ZNeg[C]] {}

    implicit def diff8[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : SumAux[A, B, C], ev2: LT[B,A]) = new ZDiffAux[ZNeg[A], ZPos[B], ZNeg[C]] {}

    implicit def diff9[A <: Nat, B <: Nat, C <: Nat]
    (implicit ev : SumAux[B, A, C], ev2: LT[A,B]) = new ZDiffAux[ZNeg[A], ZPos[B], ZPos[C]] {}
}

trait ToZInt[Z <: ZInt] {
    def apply() : Int
}

object ToZInt {
    implicit def toPosInt[N <: Nat] (implicit i: ToInt[N]) = new ToZInt[ZPos[N]] {
        def apply() = i.apply()
    }
    implicit def toNegInt[N <: Nat] (implicit i: ToInt[N]) = new ToZInt[ZNeg[N]] {
        def apply() = - i.apply()
    }
}

Some useful definitions are in object DimValOps.

object DimValOps {
    type Z0 = ZPos[_0]
    type Z1 = ZPos[Succ[_0]]
    type Z2 = ZPos[Succ[Succ[_0]]]
    type Z3 = ZPos[Succ[Succ[Succ[_0]]]]
    type Z4 = ZPos[Succ[Succ[Succ[Succ[_0]]]]]
    type Z_1 = ZNeg[Succ[_0]]
    type Z_2 = ZNeg[Succ[Succ[_0]]]
    type Z_3 = ZNeg[Succ[Succ[Succ[_0]]]]

    type length = DimVal[Z1, Z0, Z0, Z0]
    type mass = DimVal[Z0, Z1, Z0, Z0]
    type time = DimVal[Z0, Z0, Z1, Z0]
    type current = DimVal[Z0, Z0, Z0, Z1]
    type charge = DimVal[Z0, Z0, Z1, Z1]
    type area = DimVal[Z2, Z0, Z1, Z0]
    type volume = DimVal[Z3, Z0, Z1, Z0]
    type velocity = DimVal[Z1, Z0, Z_1, Z0]
    type acceleration = DimVal[Z1, Z0, Z_2, Z0]
    type momentum = DimVal[Z1, Z1, Z_1, Z0]
    type angular_momentum = DimVal[Z2, Z1, Z_1, Z0]
    type frequency = DimVal[Z0, Z0, Z_1, Z0]
    type force = DimVal[Z1, Z1, Z_2, Z0]
    type pressure = DimVal[Z_1, Z1, Z_2, Z0]
    type viscosity = DimVal[Z_1, Z1, Z_1, Z0]
    type energy = DimVal[Z2, Z1, Z_2, Z0]
    type power = DimVal[Z2, Z1, Z_3, Z0]
    type potential = DimVal[Z2, Z1, Z_3, Z_1]
    type capacitance = DimVal[Z_2, Z_1, Z4, Z2]
    type resistance = DimVal[Z2, Z1, Z_3, Z_2]
    type conductance = DimVal[Z_2, Z_1, Z3, Z2]
    type magnetic_flux = DimVal[Z2, Z1, Z_2, Z_1]
    type magnetic_flux_density = DimVal[Z0, Z1, Z_2, Z_1]
    type inductance = DimVal[Z2, Z1, Z_2, Z_2]

    val m = new length(1) //meter
    val kg = new mass(1) //kilogram
    val s = new time(1) //second
    val A = new current(1) //Ampere
    val C = new charge(1) //Coulomb
    val N = new force(1) //Newton
    val Hz = new frequency(1) //Hertz
    val Pa = new pressure(1) //Pascal
    val ohm = new resistance(1)
    val V = new potential(1) //Volt
    val S = new conductance(1) //Siemens
    val W = new power(1) //Watt
    val J = new energy(1) //Joule
    val F = new capacitance(1) //Farad
    val H = new inductance(1) //Henry
    val Wb = new magnetic_flux(1) //Weber
    val T = new magnetic_flux_density(1) //Tesla

    implicit def numToDim(d : Double) : DimVal[Z0,Z0,Z0,Z0] = new DimVal[Z0,Z0,Z0,Z0](d)
}

On the one hand, it looks frightening. On the other hand, it’s just a few lines of code to add a capability that is missing since forever. Here is some putting it through its paces.

    val c0 = new charge(4)
    println("c0="+c0.asString())

    val v = 3 * m / s
    println("v="+v.asString)

    val mom = 10 * kg * v
    println("mom="+mom.asString())

    val e = mom * v
    println("e="+e.asString())

    val p = e / (3 * s)
    println("p="+p.asString())

Which outputs:

c0=4.0 C
v=3.0 m·s-1
mom=30.0 m·kg·s-1
e=90.0 J
p=30.0 W

Look what happens when you try to compile an assignment with mismatched dimensions.

    val p : pressure = e / (3 * s)
Error:(37, 26) type mismatch;
 found   : ShapelessExperiment.DimVal[ShapelessExperiment.ZPos[shapeless.Succ[shapeless.Succ[shapeless.Nat._0]]],ShapelessExperiment.ZPos[shapeless.Succ[shapeless.Nat._0]],ShapelessExperiment.ZNeg[shapeless.Succ[shapeless.Succ[shapeless.Succ[shapeless.Nat._0]]]],ShapelessExperiment.ZPos[shapeless.Nat._0]]
 required: ShapelessExperiment.DimValOps.pressure
    (which expands to)  ShapelessExperiment.DimVal[ShapelessExperiment.ZNeg[shapeless.Succ[shapeless.Nat._0]],ShapelessExperiment.ZPos[shapeless.Succ[shapeless.Nat._0]],ShapelessExperiment.ZNeg[shapeless.Succ[shapeless.Succ[shapeless.Nat._0]]],ShapelessExperiment.ZPos[shapeless.Nat._0]]
    val p : pressure = e / (3 * s)
                         ^

Ok, the message is not entirely simple but, if you understand the type signatures, it is clear how to read it. In conclusion, even though Scala’s compile-time metaprogramming has been said to imitate, rather than implement, dependent types, you can see it is able to quack like a duck and walk like a duck. And it is perfectly capable of stepping in when Java proves too little, even though it cannot overcome fundamental limitations of the JVM.

We don’t need no stinkin’ CMS

This post is more playful than usual but it explores the question of how to achieve most of what one wants a CMS for with a few lines of Javascript and a few lines of server-side code. I’ve coined the term “Source Edit and Content Selection” for this functionality, but this was done primarily to permit me to make a pun that only viewers of a relatively unknown teen movie will understand. Let’s move on and please try to remember the safe word!

What is Wunder-SECS

Wunder-SECS is a framework that does two things:

  • Lets the user edit the placeholders marked in a page, from within the page itself, by specifying assets to fill them or by directly setting content in them,
    and also by setting style properties on elements of the page.
  • Lets the user save the finished page, which, from then on, is independent from Wunder-SECS and can be served by the server and rendered by any browser.

Wunder-SECS uses common Web 2.0 libraries and requires very little effort to work with sites built with standard HTML and CSS, either static or dynamic.

How Wunder-SECS composes pages

Marking an element to be editable (to be a placeholder) is a simple matter of adding a number of CSS classes to it. The mandatory class to add is editable. Other classes distinguish
different kinds of editing required, notably asset selection, of different kinds, or content editing. Wunder-SECS will recover all editable content in the page and apply the necessary
code to it.

Marking for Drag & Resize

Marking for Drag & resize is done with the draggable and resizeable CSS classes, which can also be used in combination.

Marking for Asset Selection

Marking for Asset Selection is done with a number of marker classes, which correspond to the asset folders where the assets reside.
The example below marks the element as a placeholder for assets of kind Flash, which also corresponds to the name of the folder. The placeholder is also draggable
and resizeable.

<h1 id="FlashID-container" class="editable _Flash draggable resizeable"/>

Wunder-SECS makes use of a JSON file called makeup.txt. Inside this file, there are directives that load assets or set content on placeholders, and
other directives that set style properties on elements of the page. This file is shared by all pages in a mini-site which are supposed to share look-and-feel.

Placeholder
An HTML element of the page that will be filled by Wunder-SECS, at page-load time. Any element can be a placeholder but, in order for the editing
facility to be used, the element needs to be marked (cf. Marking for Editing).
Asset
An HTML fragment that will be used to fill a placeholder. Can optionally contain its own inline <style> so that it be self-contained. Such interchangeable fragments
are typically created by Web designers, and need to be placed in asset folders with appropriate names.
Content
An HTML fragment (a.k.a. rich text) that was provided on-the-fly by the page editor. It is not reusable, since it cannot be used in other sites, and contains presentational
markup only.
Style properties
Any CSS property on any element that is useful to change in order to effect a change on a page. An example could be properties that need to be set to various elements in order to change
the page background in a meaningful way.

Editing the page

While entering edit mode, a splash screen is displayed briefly.

w-splash

When working in an editing session, you’ll see the Wunder-SECS toolbar in the page. The toolbar shows a select menu for the theme of the page.

w-toolbar

When you double-click on an asset placeholder, the toolbar will also show an asset select menu of the appropriate kind.

w-toolbar2

Selecting an asset from the select menu has an immediate effect.

w-asset-edit1
w-asset-edit2
w-asset-edit3

Double-clicking on a content placeholder, on the other hand, opens up the jWysiwyg editor.

w-wysiwyg

Placeholders are outlined by colored borders, sometimes two of them, to mark the placeholder and its being draggable at the same time. A resizeable placeholder has the familiar bottom-right handle, to
drag with the mouse, and also draggable bottom and right borders.

w-placeholder

The toolbar is also draggable, so you can move it around if it gets in the way.

Marking for Editing

Marking an element to be editable (to be a placeholder) is a simple matter of adding a number of CSS classes to it. The mandatory class to add is editable. Other classes distinguish
different kinds of editing required, notably asset selection, of different kinds, or content editing. Wunder-SECS will recover all editable content in the page and apply the necessary
code to it.

Marking for Drag & Resize

Marking for Drag & resize is done with the draggable and resizeable CSS classes, which can also be used in combination.

Marking for Asset Selection

Marking for Asset Selection is done with a number of marker classes, which correspond to the asset folders where the assets reside.
The example below marks the element as a placeholder for assets of kind Flash, which also corresponds to the name of the folder. The placeholder is also draggable
and resizeable.

<h1 id="FlashID-container" class="editable _Flash draggable resizeable"/>

Marking for Content Editing

Marking for Content Editing is done with the _Wysiwyg CSS class.
The example below marks the element as a placeholder for formatted text content.

<h1 id="text-container" class="editable _Wysiwyg"/>

Saving the page

When you are ready to save the finished page, you use one of the two “save” buttons.

w-toolbar3

Button “Save” saves the HTML and CSS that you have specified, while button “Save ESI” does the same thing but inside ESI markup. When not running inside a CDN that understands ESI
markup, the page is rendered essentially like before. When ESI markup is available and is interpreted by a CDN, then one can control the cache lifetime, expiration and upgrade of the individual assets inside a page through the facilities offered by the CDN.

<esi:include src="... asset URL ..." />
<esi:remove>... asset contents at time of save ...</esi:remove>

Integrating WunderSECS

To integrate WunderSECS in a page, you need to import the wundersecs.js file. It is written using the module pattern and exposes the WunderSECS global object. If you’re using Require.js or any other dependency manager, you might want to modify it to return the object instead of setting it at global scope.

It has a number of dependencies, which are listed below. If you rely on Require.js or any other dependency manager, remember to codify interdependencies within that list
correctly.

<script src="/appropriate_path/jquery-1.8.3.js" type="text/javascript"></script>
<script src="/appropriate_path/jquery.ui.core.js" type="text/javascript"></script>
<script src="/appropriate_path/jquery.ui.position.js" type="text/javascript"></script>
<script src="/appropriate_path/jquery.ui.widget.js" type="text/javascript"></script>
<script src="/appropriate_path/jquery.ui.mouse.js" type="text/javascript"></script>
<script src="/appropriate_path/jquery.ui.draggable.js" type="text/javascript"></script>
<script src="/appropriate_path/jquery.ui.resizable.js" type="text/javascript"></script>
<script src="/appropriate_path/jquery.ui.selectmenu.js" type="text/javascript"></script>
<script src="/appropriate_path/jquery.validate-1.9.0.min.js" type="text/javascript"></script>
<script src="/appropriate_path/jquery-center.js" type="text/javascript"></script>
<script src="/appropriate_path/wundersecs/wundersecs.js" type="text/javascript"></script>
<script src="/appropriate_path/wysiwyg/jquery.wysiwyg.js" type="text/javascript"></script>
<script src="/appropriate_path/jquery.jeditable.mini.js" type="text/javascript"></script>
<script src="/appropriate_path/wysiwyg/jquery.jeditable.wysiwyg.js" type="text/javascript"></script>
<link href="/appropriate_path/jquery.ui.selectmenu.css" type="text/css" rel="stylesheet" />
<link href="/appropriate_path/jquery-ui-1.8.18-smoothness.css" type="text/css" rel="stylesheet" />
<link href="/appropriate_path/jquery.ui.theme.css" type="text/css" rel="stylesheet" />
<link href="/appropriate_path/wysiwyg/jquery.wysiwyg.css" type="text/css" rel="stylesheet" />

At normal rendering time, you don’t need to do anything. The page is self-contained.

If this is an editing session (which could be conveyed to the page using a parameter, or a cookie, or any other way), then you must call the initEdit function, in order
for edit functionality to be setup, and the WunderSECS toolbar to appear. You must pass the value of the “save” HTTP parameter, which is used by the “save” buttons.

WunderSECS.initEdit(saveFlag);

This is all that needs to be done at the client. At the server, right now the code expects a particular implementation of the AJAX call (WunderSecsEdit.ashx) for editing the makeup, but the specific URL used can be exposed as an initialization option, so as to be able to use it in any application.

Future enhancements might be:

  • Generalizing the background functionality to cover any combination of CSS.
  • Enabling recursive assets, where an asset might have placeholders of its own.

The code for wundersecs.js follows.

var WunderSECS = (function ($) {
    var my = {};
    var makeupData = {};

    function loadFragment(selector, url) {
        if ($(selector).hasClass('_Wysiwyg')) {
            $(selector).html(url);
        } else {
            $(selector).addClass("loading");
            $.get(url, function (data) {
                if (my.saveFlag == "esi") {
                    var esi = $('<esi:include src="' + url + '"/>');
                    $(selector).html(esi);
                    var rem = $('<esi:remove>');
                    $(rem).html(data);
                    $(selector).append(rem);
                } else {
                    $(selector).html(data);
                }
                $(selector).removeClass("loading");
            }, 'html');
        }
    }

    function startsWith(s, str) {
        return s.slice(0, str.length) == str;
    }

    function makeEditable(e) {
        var kind = '';
        if ($(e).hasClass('_Wysiwyg')) {
            var isResizeable = $(e).hasClass("ui-resizable");
            if (isResizeable) {
                $(e).resizable("destroy");
            }
            var wasResizeable;
            $(e).editable(function (val) {
                //                                var html = $.parseHTML(val);
                //                                $(html).remove('.ui-resizable-handle');
                //                                val = $('<div/>').append($(html)).html();

                $.post("/WunderSecsEdit.ashx", {
                    path: window.location.pathname,
                    key: '#' + $(e).attr('id'),
                    value: JSON.stringify(val)
                });
                if (isResizeable) {
                    makeResizeable($(e));
                }
                return val;
            }, {
                type: "wysiwyg",
                event: "dblclick",
                submit: "OK",
                cancel: "Cancel",
                onblur: "ignore"
                , onedit: function () {
                    wasResizeable = $(e).hasClass("ui-resizable");
                    if (wasResizeable) {
                        $(e).resizable("destroy");
                    }
                    return true;
                },
                onreset: function () {
                    if (wasResizeable) {
                        makeResizeable($(e));
                    }
                    return true;
                }
            });
            if (isResizeable) {
                makeResizeable($(e));
            }
            return;
        }
        if ($(e).hasClass('_Moto')) kind = 'Moto';
        if ($(e).hasClass('_TC')) kind = 'TC';
        if ($(e).hasClass('_MsisdnEntry')) kind = 'MsisdnEntry';
        if ($(e).hasClass('_Flash')) kind = 'Flash';
        if ($(e).hasClass('_Sub')) kind = 'Subtitle';
        if ($(e).hasClass('_Terms')) kind = 'SM';
        if ($(e).hasClass('_Button')) kind = 'Button';

        $(e).bind("dblclick", function () {
            $.get('/cms/assets/' + kind + '/files.txt', function (data) { //testing
                var selectlist = $.parseHTML(data);
                $('#editplaceholder').show();
                $('#editlist').html(selectlist);
                $(selectlist).selectmenu({
                    width: 200,
                    select: function (event, options) {
                        $.ajax({
                            type: "GET",
                            url: options.value,
                            success: function (value) {
                                makeupData[$(e).attr('id')] = options.value;
                                $.post("/WunderSecsEdit.ashx", {
                                    path: window.location.pathname,
                                    key: '#' + $(e).attr('id'),
                                    value: JSON.stringify(options.value)
                                });
                                var isDraggable = $(e).hasClass("ui-draggable");
                                if (isDraggable) {
                                    $(e).draggable("destroy");
                                }
                                var isResizeable = $(e).hasClass("ui-resizable");
                                if (isResizeable) {
                                    $(e).resizable("destroy");
                                }
                                $(e).html(value);
                                if (isDraggable) {
                                    makeDraggable($(e));
                                }
                                if (isResizeable) {
                                    makeResizeable($(e));
                                }
                                $('#editplaceholder').hide();
                                $('#editlist').html('');
                            }
                        });
                    }
                });
            }, 'html');
        });
    }

    function makeDraggable(e) {
        $(e).draggable({
            stop: function (event, ui) {
                var top = ui.position.top;
                var left = ui.position.left;
                //alert("Top " + top + ", left " + left);
                $.post("/WunderSecsEdit.ashx", {
                    path: window.location.pathname,
                    key: "position.#" + $(e).attr('id'),
                    value: JSON.stringify({ top: top, left: left })
                });

            }
        });
    }

    function makeResizeable(e) { //todo: look at the CSS and decide which dimensions should be resizeable
        $(e).resizable({
            stop: function (event, ui) {
                var width = ui.size.width;
                var height = ui.size.height;
                $.post("/WunderSecsEdit.ashx", {
                    path: window.location.pathname,
                    key: "size.#" + $(e).attr('id'),
                    value: JSON.stringify({ width: width, height: height })
                });

            }
        });
    }

    function makeEditableBg() {
        $.get('/cms/assets/bg/files.txt', function (data) { //testing
            var selectlist = $.parseHTML(data);
            $('#editbg').html(selectlist);
            $(selectlist).selectmenu({
                width: 200,
                select: function (event, options) {
                    var url = options.value.split('|');
                    makeupData.bg = url;
                    $.post("/WunderSecsEdit.ashx", {
                        path: window.location.pathname,
                        key: "bg",
                        value: JSON.stringify(url)
                    });
                    $('body').css("background", url[0]);
                    $('.oneColFixCtr #container').css("background", url[1]);
                    $('.oneColFixCtr #container .enterSomething.step1').css("background", url[2]);
                }
            });
        }, 'html');
    }

    function showSplash() {
        $('#wundersecs-splash').center().show().css("z-index", 1).css("-moz-box-shadow", "0 0 10px 10px #ffffff").css("-webkit-box-shadow", "0 0 10px 10px #ffffff").css("box-shadow", "0 0 10px 10px #ffffff");
        $('#wundersecs-splash').fadeIn(1000).delay(2000).fadeOut(1000);
    }

    function savePage() {
        var ready = $('.loading').length == 0;
        if (ready) {
            savePageInner();
            return;
        }
        setTimeout(function () {
            savePage();
        }, 1000);
    }

    function savePageInner() {
        var pageHtml = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\n<html xmlns="http://www.w3.org/1999/xhtml">\n' + $('html').html() + '\n</html>';
        var fullurl = window.location.href.replace(/save=[^&]*/g, '');
        var url = window.location.pathname;
        $.ajax("/WunderSecsEdit.ashx?path=" + encodeURIComponent(url), {
            type: "PUT",
            mimeType: "text/html",
            data: pageHtml,
            success: function (data, textStatus, jqXHR) {
                window.location.href = fullurl;
            }
        });
    }

    function compose() {
        $.get('makeup.txt', function (data) {
            makeupData = data;
            for (var key in data) {
                var url = data[key];
                //$(key).load(url);
                if (key == "bg") {
                    $('body').css("background", url[0]);
                    $('.oneColFixCtr #container').css("background", url[1]);
                    $('.oneColFixCtr #container .enterSomething.step1').css("background", url[2]);
                } else if (startsWith(key, "position.")) {
                    key = key.slice("position.".length, key.length);
                    $(key).css(url);
                } else if (startsWith(key, "size.")) {
                    key = key.slice("size.".length, key.length);
                    $(key).css(url);
                } else {
                    loadFragment(key, data[key]);
                }
            }
            if (my.saveFlag) savePage();
        }, 'json');
    };

    my.initEdit = function (saveFlag) {
        my.saveFlag = saveFlag;
        compose();

        if (my.saveFlag) return;

        $.get('/stat/wundersecs/wundersecs-toolbar.htm', function (data) {
            var toolbar = $.parseHTML(data);
            $('body').append(toolbar);
            $(toolbar).find('button.savebutton').bind("click", function () {
                window.location.href = window.location.href + '&save=true';
            });
            $(toolbar).find('button.saveesibutton').bind("click", function () {
                window.location.href = window.location.href + '&save=esi';
            });
            $(toolbar).draggable();
        }, 'html');
        $.get('/stat/wundersecs/wundersecs-style.htm', function (data) {
            var style = $.parseHTML(data);
            $('head').append(style);
        }, 'html');
        $.get('/stat/wundersecs/wundersecs-splash.htm', function (data) {
            var splash = $.parseHTML(data);
            $('body').append(splash);
            showSplash();
        }, 'html');

        $('.draggable').each(function (i, e) {
            makeDraggable(e);
        });
        $('.resizeable').each(function (i, e) {
            makeResizeable(e);
        });
        $('.editable').each(function (i, e) {
            makeEditable(e);
        });
        makeEditableBg();

    };

    return my;
})($);

And this might be an implementation for WunderSecsEdit.ashx, purely for demonstration. Feel free to substitute your own.

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Web;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;

namespace Controller
{
    /// <summary>
    /// Summary description for WunderSecsEdit
    /// </summary>
    public class WunderSecsEdit : IHttpHandler
    {

        public void ProcessRequest(HttpContext context)
        {
            var path = context.Request["path"];
            if (context.Request.HttpMethod == "POST" && path!=null)
            {
                var key = context.Request["key"];
                var value = context.Request["value"];
                if (key != null && value != null)
                {
                    var filepath = path.Substring(0, path.LastIndexOf('/')) + "/makeup.txt";
                    var mappedPath = context.Server.MapPath(filepath);
                    var jsonSerializer = new JsonSerializer();
                    var streamReader = new StreamReader(mappedPath);
                    var makeup = jsonSerializer.Deserialize(new JsonTextReader(streamReader));
                    streamReader.Close();
                    if (makeup is JObject)
                    {
                        var jObject = (JObject) makeup;
                        var jvalue = jsonSerializer.Deserialize(new JsonTextReader(new StringReader(value)));
                        if (jvalue is JToken)
                        {
                            jObject[key] = (JToken) jvalue;
                            var streamWriter = new StreamWriter(mappedPath);
                            jsonSerializer.Serialize(new JsonTextWriter(streamWriter), jObject);
                            streamWriter.Close();
                            //context.Response.ContentType = "text/plain";
                            //context.Response.Write("Hello World");
                        }
                        if (jvalue is string)
                        {
                            jObject[key] = (string) jvalue;
                            var streamWriter = new StreamWriter(mappedPath);
                            jsonSerializer.Serialize(new JsonTextWriter(streamWriter), jObject);
                            streamWriter.Close();
                            //context.Response.ContentType = "text/plain";
                            //context.Response.Write("Hello World");
                        }
                    }
                }
            }
            else if (path != null && context.Request.HttpMethod == "PUT")
            {
                context.Request.InputStream.Position = 0;
                var reader = new StreamReader(context.Request.InputStream);
                string payload = reader.ReadToEnd();
                var mappedPath = context.Server.MapPath(path);
                using (var streamWriter = new StreamWriter(mappedPath))
                {
                    streamWriter.Write(payload);
                    streamWriter.Close();
                }
            }
        }

        public bool IsReusable
        {
            get
            {
                return false;
            }
        }
    }
}

Lamino, or What the Heck Do I Know About Dependency Injection

Recently, someone asked me if I knew what Dependency Injection was. Well, Dependency Injection and I go way back, I have to say. Or Inversion Of Control. Or whatever it was called at the time, I can’t remember. The time, however, was shortly after the turn of the centrury. Enter nostalgic guitar, please.

I was working at Velti, building a system for SMS interactions. Not content with the state of configuration management, I had developed a rudimentary facility to instantiate objects and set their properties. Today, you’d call it a DI/IoC container. Because reinventing the wheel is always fun, but not always productive, as we were growing this facility I was looking around for other similar stuff and for people with similar ideas. There weren’t any, except for Rod Johnson and his book “Expert one-on-one J2EE design and development” describing something called Interface21. But it was enough. The approach Rod was describing was very much to my liking. I immediately switched the XML grammar to that of what came to be known as Spring and adopted most of its concepts.

However, I did not switch to Spring, because it lacked two things, both of which we had come to rely upon.

  • One of them was the ability to separate the DI part from the instantiation part. We were using XmlWeb2, which I might blog about someday, which was instantiating MVC actions and views using a reloading classloader, so we needed these two phases to be separate. (BTW, it’s very easy to write a reloading classloader in Java, once you realize the following: a classloader can never reload a class!).
  • The other was a mechanism to build configuration in layers. It it mostly the latter that the rest of this post will be occupied with. Or rather its descendant, which is Lamino.

Lamino

The blurb: Lamino is a high-level configuration metalanguage that enhances basic IoC (Inversion of Control a.k.a. Dependency Injection) with configuration layering and encapsulated components. It is hosted by an IoC language and it “compiles” to it. Using configuration layering, the application developer can create editions or extensions by expressing the differences of each configuration layer from the layer below it. Using encapsulated components, the application developer can define reusable configuration chunks that import and export specific services, while hiding internal contents.

You should understand that, in my puritan mind, there is no program structure outside of the configuration. The whole object graph of the singletons that comprise the logic layer, should be there. That object graph and the collaborations it describes is the application. Configuring a different edition or deployment can involve a lot more than modifying some properties. A whole portion of the object graph could need be redefined. That’s why relying on a separate layer of property files, or something, can only be half a solution. And doing a different build of a system just to change how some classes collaborate, is not a solution at all.

I will not replicate the information in the project Web site here. But I will show you what Lamino does by walking you through the examples in the distribution, which is not covered by the site. They are the same for all three containers covered (you’d better get the latest files from SVN (here, the DotNet distribution), by the way, because the WebContextHandler for .Net is only to be found there). WordPress insists on mangling large XML snippets containing “object” elements, so you’ll have to view the files from there.

Example

To cut a long story short the resulting system can be found in Derived-derived.xml, which is the actual output of Lamino. To make the point that Lamino is purely transformation and did not need a full-blown programming language to implement it, it is written in XSLT (which, BTW, is the first mainstream functional language).

Modify an object with extends=”extends”

Let’s have a look at object “ObjectToBeExtendedFromMoreBase”, keeping in mind that element “x-object” is really “object” but I had to rename it to keep WordPress from slurping it.

This object is defined in MoreBase.xml:

	<x-object id="ObjectToBeExtendedFromMoreBase">
		<property name="PropertyToBeCopiedAsIs" value="foo"/>
		<property name="PropertyToBeReplaced" value="value to be replaced"/>
		<property name="PropertyToBeRemoved" value="foo"/>
		<property name="PropertyToBeAppendedTo">
			<list>
				<value>val1</value>
				<value>val2</value>
			</list>
		</property>
		<property name="PropertyToBePrependedTo">
			<list>
				<value>val1</value>
				<value>val2</value>
			</list>
		</property>
	</x-object>

And extended in Base.xml using extends=”extends”.

	<x-object id="ObjectToBeExtendedFromMoreBase" extends="extends">
		<property name="NewProperty" value="foo2"/>
		<property name="PropertyToBeReplaced" combine="replace" value="new value"/>
		<property name="PropertyToBeRemoved" combine="remove"/>
		<property name="PropertyToBeAppendedTo" combine="append">
			<list>
				<value>newval1</value>
				<value>newval2</value>
			</list>
		</property>
		<property name="PropertyToBePrependedTo" combine="prepend">
			<list>
				<value>newval1</value>
				<value>newval2</value>
			</list>
		</property>
	</x-object>

The resulting configuration is the following. Note that I couldn’t coerce either MSXML or Xalan to produce whitespace to my liking, so no whitespace remains in the output. Lamino annotates the output with some comments to make it easier to trace the output.

<!--Extended-->
<x-object id="ObjectToBeExtendedFromMoreBase" scope="application" visibility="public" dependency-check="default" singleton="true" abstract="false" lazy-init="default" autowire="default">
<property name="NewProperty" value="foo2"></property>
<!--Replaced-->
<property name="PropertyToBeReplaced" value="new value"></property>
<!--Property PropertyToBeRemoved was removed-->
<property name="PropertyToBeAppendedTo">
<list>
<!--Original elements-->
<value>val1</value>
<value>val2</value>
<!--Appended elements-->
<value>newval1</value>
<value>newval2</value>
</list>
</property>
<property name="PropertyToBePrependedTo">
<list>
<!--Prepended elements-->
<value>newval1</value>
<value>newval2</value>
<!--Original elements-->
<value>val1</value>
<value>val2</value>
</list>
</property>
<property name="PropertyToBeCopiedAsIs" value="foo"></property>
</x-object>

Property “NewProperty” is added in Base.xml.

Property “PropertyToBeReplaced” starts off with value=”value to be replaced”, but this value is overriden using combine=”replace” in Base.xml with value=”new value”, which is the value that remains in the end.

Property “PropertyToBeRemoved” is removed using combine=”remove” in Base.xml.

Property “PropertyToBeAppendedTo” contains two elements to begin with, and two more are added in Base.xml using combine=”append”. A similar modification is done to property “PropertyToBePrependedTo”, where two elements are prepended using combine=”prepend” in Base.xml.

Finally, property “PropertyToBeCopiedAsIs” is not modified at all.

Replace an object with extends=”overrides”

Let’s focus now on object “ObjectToBeReplacedFromMoreBase”.

	<x-object id="ObjectToBeReplacedFromMoreBase">
		<property name="Property2" value="bar to be replaced"/>
	</x-object>

Things are simpler in this case, as this object is overriden in Base.xml.

	<x-object id="ObjectToBeReplacedFromMoreBase" extends="overrides">
		<property name="Property2" value="new value"/>
	</x-object>

The resulting object is, indeed, what Base.xml prescribes.

<!--Overriden-->
<x-object id="ObjectToBeReplacedFromMoreBase" scope="application" visibility="public" dependency-check="default" singleton="true" abstract="false" lazy-init="default" autowire="default">
<property name="Property2" value="new value"></property>
</x-object>

Importing components

Now, let’s look how one can define DI components. Derived.xml imports the same component twice.

	<import resource="Component.xml"/>
	<import resource="Component.xml">
		<hiding object="OtherPublicComponentToBeHidden"/>
		<renaming object="ExplicitlyPublicComponent" as="Compo1"/>
		<renaming object="ImplicitlyPublicComponent" as="Compo2"/>
		<providing object="NewObject" as="External"/>
	</import>

Note that the component itself employs configuration inheritance in its definition.

ComponentBase.xml:

<x-objects xmlns="http://www.springframework.net"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.net
lamino-objects.xsd">
	<x-object id="ExplicitlyPublicComponent" visibility="public">
		<property name="Property3" value="foobar original"/>
		<property name="Property4">
			<ref local="Hidden"/>
		</property>
	</x-object>
	<x-object id="ImplicitlyPublicComponent">
		<property name="Property6" value="foobaz original"/>
		<property name="Property7" ref="External"/>
	</x-object>
	<x-object id="OtherPublicComponentToBeHidden"/>
	<x-object id="Hidden" visibility="private">
		<property name="Property5" value="baz original"/>
	</x-object>
</x-objects>

Component.xml:

<x-objects xmlns="http://www.springframework.net"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.net
lamino-objects.xsd" extends="ComponentBase.xml">
	<x-object id="ExplicitlyPublicComponent" visibility="public" extends="extends">
		<property name="Property3" value="foobar" combine="replace"/>
	</x-object>
	<x-object id="Hidden" visibility="private" extends="extends">
		<property name="Property5" value="baz" combine="replace"/>
	</x-object>
</x-objects>

As you see, the first importing does not specify anything in particular, while the second importing specifies the following.

  • Object “OtherPublicComponentToBeHidden” is to be hidden.
  • Objects “ExplicitlyPublicComponent” and “ImplicitlyPublicComponent” are to be renamed.
  • Object “NewObject” is to replace references to object “External”.

The effects of these (and the comparison to the unmodified component import) is seen in the output.

<x-object id="ExplicitlyPublicComponent" scope="application" dependency-check="default" singleton="true" abstract="false" lazy-init="default" autowire="default">
<property name="Property3" value="foobar"></property>
<property name="Property4">
<ref local="Hidden-IDAQJLVIDASJLV"></ref>
</property>
</x-object>
<x-object id="ImplicitlyPublicComponent" scope="application" dependency-check="default" singleton="true" abstract="false" lazy-init="default" autowire="default">
<property name="Property6" value="foobaz original"></property>
<property name="Property7" ref="External"></property>
</x-object>
<x-object id="OtherPublicComponentToBeHidden" scope="application" dependency-check="default" singleton="true" abstract="false" lazy-init="default" autowire="default"></object>
<x-object id="Hidden-IDAQJLVIDASJLV" scope="application" dependency-check="default" singleton="true" abstract="false" lazy-init="default" autowire="default">
<property name="Property5" value="baz"></property>
</x-object>
<x-object id="Compo1" scope="application" dependency-check="default" singleton="true" abstract="false" lazy-init="default" autowire="default">
<property name="Property3" value="foobar"></property>
<property name="Property4">
<ref local="Hidden-IDAKJLVIDASJLV"></ref>
</property>
</x-object>
<x-object id="Compo2" scope="application" dependency-check="default" singleton="true" abstract="false" lazy-init="default" autowire="default">
<property name="Property6" value="foobaz original"></property>
<property name="Property7" ref="NewObject"></property>
</x-object>
<x-object id="OtherPublicComponentToBeHidden-IDAXDEUIDAYDEU" scope="application" dependency-check="default" singleton="true" abstract="false" lazy-init="default" autowire="default"></object>
<x-object id="Hidden-IDAKJLVIDASJLV" scope="application" dependency-check="default" singleton="true" abstract="false" lazy-init="default" autowire="default">
<property name="Property5" value="baz"></property>
</x-object>

Note that the “visibility” attribute, meant to be used inside components, specifies whether the object will be publicly visible in the output. Hiding is done via a simple renaming of IDs (here, to “Hidden-IDAQJLVIDASJLV” and “Hidden-IDAKJLVIDASJLV”).

Importing
The component expects to find a reference to object “External”. The first importing does not modify this specification and resolution to that reference is done using object “External” defined in Derived.xml. Note that Lamino does not try to verify the validity or even the syntax of the configuration. If object “External” was a broken reference, it would stay a broken reference.

The second importing modifies the reference to demand an object named “NewObject”, also defined in Derived.xml. Same caveat as before regarding validity of references.

Exporting and Hiding
The component exports all objects that are explicitly or implicitly public. These are objects “ExplicitlyPublicComponent”, “ImplicitlyPublicComponent” and “OtherPublicComponentToBeHidden”, exporting with these names by the first, unmodified, importing. The second importing renames the first two objects and hides the third one. Had it not hidden it, the resulting configuration would contain two objects with the same ID, which would be… inconvenient but, as I have said, Lamino does not check your Spring.Net grammar and syntax (same for Spring for Java and Castle Windsor).

Embedding in Web.config

As my latest jobs where in .Net shops, I have paid particular attention to the Spring.Net distribution. In the latest code to be found in SVN, there is a WebContextHandler to help integrate Lamino in a Web application, in a context to be used instead of the one Spring.Net provides.

using System;
using System.Collections.Generic;
using System.Xml.Xsl;
using System.Linq;
using System.Web;
using Spring.Context;
using WCH=Spring.Context.Support.WebContextHandler;

namespace Lamino.Spring
{
    public class WebContextHandler : WCH 
    {
        protected override IApplicationContext InstantiateContext(IApplicationContext parent, object configContext, string contextName, Type contextType, bool caseSensitive, string[] resources)
        {
            var newResources = LaminoExtend(resources);
            return base.InstantiateContext(parent, configContext, contextName, contextType, caseSensitive, newResources);
        }

        protected string[] LaminoExtend(string[] resources)
        {
            var httpServerUtility = HttpContext.Current.Server;
            XslCompiledTransform transform = new XslCompiledTransform();
            XsltSettings xsltSettings = new XsltSettings();
            xsltSettings.EnableDocumentFunction = true;
            transform.Load(httpServerUtility.MapPath("~/lamino-objects.xslt"),xsltSettings,null);
            var newResources = (string[])resources.Clone();
            for (int i = 0; i < newResources.Length; i++)
            {
                var finalResource = newResources[i] + ".Lamino";
                var resourceFile = httpServerUtility.MapPath(resources[i]);
                var finalResourceFile = httpServerUtility.MapPath(finalResource);
                if (!System.IO.File.Exists(finalResourceFile) || System.IO.File.GetLastWriteTime(finalResourceFile).CompareTo(System.IO.File.GetLastWriteTime(resourceFile)) < 0)
                {
                    transform.Transform(resourceFile, finalResourceFile);
                }
                newResources[i] = finalResource;
            }
            return newResources;
        }
    }
}

You can be up and running with layered and componentized DI in no time!

Configuration layering

I want to see configuration layering become standard practice. It seems to me that any program meant to be deployed in various editions and deployments can benefit from having to specify just the delta from the base. The layers can be, for example: the core, interface-less part of the application, the different embodiments of the application in specific user interfaces and particular deployments. I would be thrilled if Spring and Spring.Net implemented these concepts themselves but, until they do, I offer Lamino as a lightweight solution to that need but, most of all, I intend it to demonstrate the need itself by showing you how easy it can be.

Fundamentum

Let me start by saying that ASP.NET MVC is terrific. Before ASP.NET MVC, one had to use ASP.NET, which I hope the universe will soon forget, so moving to ASP.NET MVC is like using a tablet instead of a clay tablet (except I am overly generous towards ASP.NET with the analogy). Really. There is no reason not to use ASP.NET MVC for your Web applications – if you enter a time machine and land ten years ago, when Struts on Java was already old news.

We live in 2013 now, folks, and the Web is moving forward. So, what is a dot-net Web programmer supposed to do? This post is a particular answer to that question I gave myself a few moons ago. Note that it is a particular, idiosyncratic even, answer and is meant to implement CRUD UIs, not any Web app in general. My answer was the Fundamentum framework, the code of which I will not open-source at this time.

Fundamentum assumes your Web application is built according to the client-side MVC or MVP pattern. It currently uses Backbone.js as its underlying Javascript framework, and offers a Backbone View and a Backbone Model to extend but it could be adapted to any other similar framework. Templating is done with the Handlebars.js library.

The application has no “dirty” state after each edit, meaning, there are never data that must be “submitted” to the server with an explicit later action (although it can have “stale” data, if other users are editing the same entities).

ASP MVC 4 is currently used to provide navigation, authentication, authorization and basic layout, while the content-centric parts of each page are rendered via Backbone views. It relies on server-side support for its Ajax calls, and the current implementation contains C# utilities and base classes to use to implement ASP MVC Controllers with the needed functionality.

Declarative annotations for editing

Fundamentum allows the declarative annotation of editable content in the page, whether in the “static” or “templated” parts of the markup. Annotation is done using special classes. Later on, a full grammar for the annotations will be presented. As an example, the following annotation instructs Fundamentum to create in-place editing functionality for whatever markup is generated by the Carrier partial template, and informs Fundamentum to create specifically a “select” list for the Carrier property of the model, and use the Carrier partial template to render each “option” in the list.

<span class="edit attr:Carrier select tmpl:Carrier">{{#with Carrier}} {{> Carrier }} {{/with}}</span>

Any property in the model can be edited, however deep in the object graph. Model properties are actually specified using Spring Expressions. An example of a more complicated expression is the following, which reaches inside the PurchasePlanPayment collection to find a particular element and refers to its ChargeUnit property.

PurchasePlanPayments.^{PurchasePlanPaymentID==new System.Guid('{{ PurchasePlanPaymentID }}')}.ChargeUnit

Fundamentum allows editing of single attributes, in preference to the usual Backbone practice of updating the whole model at once. It actually supplements the default Backbone Ajax API
(the set of calls that Backbone does by default and expects the server to understand) with a new one, which updates a single property and is caused by the saveAttribute(attributeName, value, callback) model method.

PUT model-url-root / model-id / property-name HTTP/1.1
... HTTP Headers ...

JSON rendering of the new value

It also supplements the default Backbone API with a call to return available options in the “select” list for a property, like in the initial example.

GET model-url-root /OPTIONS/ model-id / property-name HTTP/1.1
... HTTP Headers ...

A third addition is the support for cloning, using the following call, caused by the createClone() model method.

POST model-url-root  / model-id /Clone HTTP/1.1
... HTTP Headers ...

Finally, there is support for adding a new element in a collection property with the following call, caused by the pushAttribute(attributeName, value, callback) model nethod.

POST model-url-root / model-id / property-name HTTP/1.1
... HTTP Headers ...

Server-side support for responding to these calls will be presented shortly. You can read about standard Backbone model methods and associated HTTP calls at Backbone Tutorials.

Edit Annotation Grammar

The essential annotation for editable content is class edit. On its own, it invokes a simple, in-line editor, as in the following.

<span class="edit attr:SubscriptionPlan.DurationNumberofUnits">{{ SubscriptionPlan.DurationNumberofUnits}}</span>

Class edit must appear together with a class which is the property expression preceded by attr: (NB. the colon character). Since the whole must be a valid class name, and will also become part of a URL in the Ajax call, the property expression must be suitably URL-escaped, as in the example of a complicated expression above, where the space character inside the expression new Guid was replaced by %20.

Class select, as we’ve seen, invokes a “select” list containing available options fetched from the server. In most cases, this must appear together with a class which is the token tmpl: (NB. the colon character) suffixed by the name of the Handlebars partial template to use in order to render each option (which is returned as a JSON object). The latter is usually the same template used to render the editable content in the first place, unless a different presentation is needed. The tmpl: class is not needed if the OPTIONS call is overriden to return a character string, which will be taken as an already formatted option HTML text.

Class datepicker invokes a date-time picker.

Class checkbox invokes a checkbox control. In the application using Fundamentum, which uses a Metro-style, tile-based interface, the checkbox control has been overriden with an actionable tile activity flag.

Finally, class wysiwyg invokes a rich-text editor (jWysiwyg).

All in-place editing is done using jEditable, with some custom additions.

Fundamentum Controllers

A Fundamentum Controller (ASP MVC Controller) is needed per basic entity, where a basic entity is any entity which is at the root of an object graph that we deem necessary to manipulate as a whole. Except for those classes which happen to be basic entities themselves, and will end up with a Controller each, any classes that represent “value entities”, which don’t have an identity separate from the basic entity from which they stem, don’t (and shouldn’t) have a Controller dedicated to them.

I found AttributeRouting a very useful notation for HTTP routes, and it is used exclusively in all Fundamentum Controllers.

In fact, I have created a special attribute, FundamentumRouteConventionAttribute, inheriting from RouteConventionAttributeBase, that annotates controllers to give them convention-based routes (ignore the parts about tags, as I’ll not elaborate on them in this post).

        private static readonly List<FundamentumRouteConventionInfo> Conventions = new List<FundamentumRouteConventionInfo>
        {
            new FundamentumRouteConventionInfo("Index", "GET", ""),
            new FundamentumRouteConventionInfo("Get", "GET", "{id}"),
            new FundamentumRouteConventionInfo("Delete", "DELETE", "{id}"),
            new FundamentumRouteConventionInfo("Update", "PUT", "{id}"),
            new FundamentumRouteConventionInfo("Create", "POST", ""),
            new FundamentumRouteConventionInfo("Clone", "POST", "{id}/Clone",9),
            new FundamentumRouteConventionInfo("SetAttributeValue", "PUT", "{id}/{AttributeName}"),
            new FundamentumRouteConventionInfo("AttributeCollectionAddNew", "POST", "{id}/{AttributeName}"),
            new FundamentumRouteConventionInfo("AttributeOptions", "GET", "OPTIONS/{id}/{AttributeName}"),
            new FundamentumRouteConventionInfo("GetTagsAvailable", "GET", "TAGS/ALL",9),
            new FundamentumRouteConventionInfo("GetTags", "GET", "TAGS/{id}"),
            new FundamentumRouteConventionInfo("RemoveTag", "DELETE", "TAGS/{id}/{Tag}"),
            new FundamentumRouteConventionInfo("AddTag", "PUT", "TAGS/{id}/{Tag}")
        };

All Fundamentum Controllers inherit from GenericEditController, which offers basic functionality for implementing Backbone and Fundamentum calls. The GenericEditController is parameterized by the class of the entity and the class of its key. We also use what AttributeRouting calls Route Conventions to create routes automatically for methods, whether inherited from the GenericEditController or implemented in a Controller itself.

public abstract class GenericEditController<TKey, TEntity> : Controller where TEntity : class
{
    public virtual JsonResult Index() { ... }
    public virtual JsonResult Get(TKey id) { ... }
    public virtual JsonResult Delete(TKey id) { ... }
    public virtual JsonResult Update(TKey id, TEntity newEntity) { ... }
    public virtual JsonResult AttributeOptions(TKey id, string attributeName) { ... }
    public virtual JsonResult SetAttributeValue(TKey id, string attributeName) { ... }
    public virtual Tuple<object,object> SetAttributeValueHelper(string attributeName, object value, TEntity obj) { ... } // Used by SetAttributeValue but overrideable on its own, as well
    public virtual JsonResult AttributeCollectionAddNew(TKey id, string attributeName) { ... }
    public virtual T ReadStreamValue<T>() { ... }
    public abstract IRepository GetRepository();
}

Index

This is a convention-based method that responds to the GET /url-root/ call, that returns all entities.

Get

This is a convention-based method that responds to the GET /url-root/id call, that returns a specific entity.

Delete

This is a convention-based method that responds to the DELETE /url-root/id call, that deletes a specific entity.

Update

This is a convention-based method that responds to the PUT /url-root/id call, that updates a specific entity wholesale.

AttributeOptions

This is a convention-based method to respond to the OPTIONS call. It handles two cases.

  • When the property mentioned is a domain entity, in which case is fetches a list of available entities from the Repository.
  • When the property mentioned is an Enum, in which case it returns the list of available values. Integer properties which correspond to Enums are annotated with the UnderlyingEnumAttribute, as in the following example.
    [UnderlyingEnum(typeof(SpGw.Enums.PurchasePlanPaymentType))]
    public virtual int PurchasePlanPaymentTypeID { get; set; }

The Controller can inherit from a number of instantiations of the following interface.

public interface IQueryableRestrictor<T, TEntity>
{
    Func<IQueryable<T>, IQueryable<T>> Restrict(TEntity baseObject);
}

In that case, the appropriate Restrict method will be called to specialize the IQueryable computed by the Repository for the specific base entity involved in the call.

The call returns a JSON object of the form { "id1" : { ...JSON rendering... }, "id2" : { ...JSON rendering... }, ... }. Each entity can be called upon to return its own id by virtue of it implementing the Identifiable interface. This might be lifted in later versions, and the responsibility passed to the Repository directly.

SetAttributeValue

This helper method responds to the setAttributeValue model method, converting as needed. It basically calls SetAttributeValueHelper to do that, so it is the latter method that one should override to perform custom conversions etc.

AttributeCollectionAddNew

This helper method responds to the pushAttributeValue model method. The Controller can inherit from a number of instantiations of the following interface.

public interface IConstructorSpecializer&amp;lt;T, TEntity&amp;gt; where TEntity: class where T:class
{
    T Construct(TEntity baseObject);
}

In that case, the appropriate Construct method will be called to create the new collection element for the specific base entity involved in the call.

GetRepository

This method returns an IRepository, which abstracts away the data layer and provides many helpful generic methods which are called by reflection to make the magick happen. It will be
exposed in a later section.

ReadStreamValue

It is simply a helper method to call JsonHelper.ReadJsonValue<T> on the request stream.

The Repository

The Repository represents the data layer of the application. I had very little to do with its innards (George was responsible for it and I hope he’ll blog about it himself) but I present it here because it’s required by Fundamentum. It enables the GenericEditController and any application Controller to perform data operations. The Repository implements interface IRepository.

    public interface IRepository
    {
        T Load<T>(object id) where T : class; //gets an object without trip to the database
        T Get<T>(object id) where T : class;
        T SaveOrUpdate<T>(T entity) where T : class;
        void Delete<T>(T entity) where T : class;
        IQueryable<T> GetAll<T>() where T : class;
        PropertyInfo FindPrimaryKeyProperty<T>() where T : class;
        T CreateEntity<T>(object id, PropertyInfo property)  where T : class;
        object ConvertedPrimaryKey<T>(object id) where T : class;
    }

It also has the following helper method, as described in the section about property options.

    List<T> ToListRestrictable<T, TEntity>(object r, TEntity baseObject) where T : class { ... }

Currently, the application using Fundamentum implements two different data layers, one with NHibernate and one with EntityFramework, to evaluate the pros and cons of each in order to arrive at the best of them.

Don’t drink the Kool-Aid

I know it’s a beverage popular with developers working with Microsoft technologies. Don’t be afraid to give your own answers and, particularly, don’t be afraid to ask your own questions. You don’t have to believe anything just because an authority says so. Remember the dictum by Isaac Newton, “If I have seen further, it is by standing on the shoulders of giants”? Don’t do the converse, epitomized in the motto in Ketil Malde’s signature:

“If I haven’t seen further, it is by standing in the footprints of giants”.