Welle 3.0 has breaking API changes in most namespaces.
All Welle public API functions that issue requests to Riak now require a client (HTTP or PBC) to be passed as an explicit argument:
(ns welle.docs.examples
(:require [clojurewerkz.welle.core :as wc]
[clojurewerkz.welle.buckets :as wb]
[clojurewerkz.welle.kv :as kv])
(:import com.basho.riak.client.http.util.Constants))
(let [conn (wc/connect)
bucket "accounts"
key "novemberain"
val {:name "Michael" :age 27 :username key}]
(wb/create conn "accounts")
;; stores data serialized as JSON
(kv/store conn bucket key val {:content-type Constants/CTYPE_JSON_UTF8})
;; fetches it back
(kv/fetch conn bucket key))Functions that take optional arguments now require them to be proper maps (and not pseudo-keywords):
;; in 2.0
(kv/store bucket key val :content-type Constants/CTYPE_JSON_UTF8)
;; in 3.0
(kv/store conn bucket key val {:content-type Constants/CTYPE_JSON_UTF8})Welle now excludes HTTPComponents dependency for Riak client and instead
uses version 4.3 which clj-http depends on.
Welle 2.0 has breaking API changes in welle.kv functions.
This is a breaking API change.
Welle 2.0 changes how Riak responses are represented as Clojure maps. Welle now will correctly preserve all vector clocks associated with multiple siblings in the response and the response itself.
This means that welle.kv/modify will work correctly and won't make sibling
explosions worse.
The most important part of the change is how responses are represented:
every response is an immutable map that has :result key as well as other
metadata keys (:has-value?, :has-siblings?, :modified?, :content-type and so on).
This contrasts with earlier versions, where results were returned directly by functions
such as welle.kv/fetch, it is now possible to destructure the response in order to
obtain the returned value:
(require '[clojurewerkz.welle.kv :as kv])
(let [{:keys [result] :as m} (kv/fetch bucket-name k :r 1)]
(comment "Do something with the result"))Here are the keys that clojurewerkz.welle.kv/fetch returns now for every response:
:result: one or more objects returned by Riak:vclock: vector clock of the response:has-siblings?: true if response has siblings:has-value?: true if response is non-empty:modified?: false when conditional GET returned a non-modified response:deleted?: true if this object has been deleted but there is a vclock for it
Welle no longer supports Clojure 1.3.
clojurewerkz.welle.counters is a new namespace that provides operations on Riak counters:
(require '[clojurewerkz.welle.counters :as wcnt])
(let [bucket-name "counters"
counter "hit-points"]
(wcnt/increment-counter bucket-name counter)
;= 1
(wcnt/fetch-counter bucket-name counter)
;= 1
(wcnt/increment-counter bucket-name counter :value 2))
;= 3
(wcnt/increment-counter bucket-name counter :value -3))
;= 0The project now depends on org.clojure/clojure version 1.6.0. It is
still compatible with Clojure 1.4 and if your project.clj depends on
a different version, it will be used, but 1.6 is the default now.
We encourage all users to upgrade to 1.6, it is a drop-in replacement for the majority of projects out there.
Welle now uses Riak Java client 1.4.x.
Cheshire has been updated to 5.3.x.
clj-http has been updated to 0.9.1.
ClojureWerkz Support has been updated to 0.20.0.
While creating a protocol buffer cluster client you can now provide hosts and ports separated by a colon, e.g. "127.0.0.1:10017". If a port is not provided, the default port will be used.
So now you can use following format:
(wc/connect-to-cluster-via-pb! ["10.0.1.2",
"10.0.1.3",
"10.0.1.4",
"10.0.1.5",
"10.0.1.6"])
as well as following format:
``` clojure
(wc/connect-to-cluster-via-pb! ["127.0.0.1:10017",
"127.0.0.1:10027",
"127.0.0.1:10037",
"127.0.0.1:10047"])Validateur is no longer a dependency of Welle.
Don't worry, they still work well together.
Automatic JSON serialization previously unconditionally converted keys to keywords. This may be a problem for some projects, because keywords are not garbage collected.
clojurewerkz.welle.conversion/*convert-json-keys-to-keywords* is a new dynamic var that
controls this behavior. When bound to false, automatic JSON serialization won't convert
keys to keywords.
clojurewerkz.welle.kv/modify is a new function that combines clojurewerkz.welle.kv/fetch and
clojurewerkz.welle.kv/store with a user-provided mutation functions. The mutation function
should take a single Riak object as an immutable map and return a modified one.
In case of siblings, a resolver should be used.
clojurewerkz.welle.kv/modify will update modification timestamp of the object.
clojurewerkz.welle.kv/modify takes the same options as clojurewerkz.welle.kv/fetch and
clojurewerkz.welle.kv/store
clojurewerkz.welle.kv/fetch, and clojurewerkz.welle.kv/store now accept a new
option: :resolver. Resolvers are basically pure functions that take a collection of
siblings and return a collection of Riak object maps.
Resolvers can be created using
clojurewerkz.welle.conversion/resolver-from which takes a function that accepts a collection
of deserialized (unless fetch was told otherwise) values and applies any conflict resolution
logic necessary.
clojurewerkz.welle.kv/fetch-one now also supports resolvers via the :resolver option.
It will raise an exception if siblings are detected and no resolver is provided.
clojurewerkz.welle.kv/fetch, clojurewerkz.welle.kv/fetch-one, clojurewerkz.welle.kv/store,
clojurewerkz.welle.kv/delete, and clojurewerkz.welle.kv/index-query now retry operations
that fail due to a network issue or any other exception.
By default, the operations will be retrier 3 times. It is possible to provide a custom
retrier using the :retrier option. Retriers can be created using
clojurewerkz.welle.conversion/retrier-from which takes a function that accepts a callable
(an operation that may need to be retried) and needs to invoke it, handling exceptions
and applying any retrying logic needed.
clojurewerkz.welle.conversion/counting-retrier produces a retrier that will retry an operation
given number of times. This is the kind of retrier Welle uses by default.
clojurewerkz.welle.kv/fetch supports a new boolean option :skip-deserialize that allows
automatic deserialization to be skipped.
Contributed by Jonas Tehler.
Welle now depends on org.clojure/clojure version 1.5.1. It is
still compatible with Clojure 1.3+ and if your project.clj depends
on a different version, it will be used, but 1.5 is the default now.
We encourage all users to upgrade to 1.5, it is a drop-in replacement for the majority of projects out there.
In eventually consistent systems such as Riak, deleted objects may
sometimes "reappear" due to concurrent modifications. Welle by
default will filter out tombstones in
clojurewerkz.welle.kv/fetch. If you want to retrieve all objects
including tombstones, pass :return-deleted-vlock as true to
clojurewerkz.welle.kv/fetch. This behavior is new in Welle 1.4.0
which uses Riak Java client 1.1.0.
Welle now uses Riak Java client 1.1.0.
Cheshire dependency has been upgraded to version 5.0.2.
Contributed by Renaud Tircher.
clojurewerkz.welle.kv/store, when used with the :return-body option, now will automatically deserialize it
the same way clojurewerkz.welle.kv/fetch and clojurewerkz.welle.kv/fetch-one do.
Suggested by Allen Johnson.
clj-http dependency has been upgraded to version 0.6.4.
Updated HTTP cluster connections to be full URLs. This change is primarily to allow users to specify alternate ports.
HTTP cluster connections now require a full url:
(require '[clojurewerkz.welle.core :as wc])
(wc/connect-to-cluster! ["http://node1:8098/riak" "http://node2:8098/riak"])Also included in this change:
clojurewerkz.welle.core/default-port removed and replaced by the following:
clojurewerkz.welle.core/default-http-portclojurewerkz.welle.core/default-pb-port
Welle now uses reasonable vclock pruning setting defaults in clojurewerkz.welle.buckets/update.
Kudos to @mefesto for reporting the issue.
clojurewerkz.welle.solr is a new namespace that provides support for Riak Search using the Solr API.
Both indexing and querying are supported:
(require '[clojurewerkz.welle.solr :as wsolr])
;; indexing
(wsolr/delete-via-query "an-index" "text:*")
(wsolr/index bucket-name {:username "clojurewerkz"
:text "Elastisch beta3 is out, several more @elasticsearch features supported github.com/clojurewerkz/elastisch, improved docs http://clojureelasticsearch.info #clojure"
:timestamp "20120802T101232+0100"
:id 1})(require '[clojurewerkz.welle.solr :as wsolr])
;; querying
(let [result (wsolr/search "an-index" "title:feature")
hits (wsolr/hits-from result)]
(println result)
(println hits))Documents stored via Riak K/V (clojurewerkz.welle.kv/store) with the content type of application/json, application/xml or text/plain will be indexed
if Riak Search is enabled for the bucket.
Welle now provides transparent serialization and deserialization support for SMILE, just like it has for
JSON, compressed data and Clojure reader. To use it, set :content-type of a value to "application/jackson-smile".
SMILE serialization can be extended to custom data types, see Cheshire documentation for more information.
Welle now uses (and depends on) Cheshire for JSON serialization. clojure.data.json is no longer a dependency.
Welle now depends on org.clojure/clojure version 1.4.0. It is still compatible with Clojure 1.3 and if your project.clj depends
on 1.3, it will be used, but 1.4 is the default now.
We encourage all users to upgrade to 1.4, it is a drop-in replacement for the majority of projects out there.
clojurewerkz.welle.kv/fetch-all is a convenience functions that retrieves multiple keys concurrently (and in parallel, on multi-core
machines), optimistically assuming there will be no siblings for each one.
Validateur, Clojure validation library that Welle depends on, was updated to 1.2.0.
Buckets with enabled search now get indexing precommit hook added automatically. Previously if the precommit hook was not added in addition, values stored in the bucket were not indexed.
clojurewerkz.welle.buckets/update doc string incorrectly listed the name of the option that enables search:
it is :enable-search, not :enabled-for-search.
clojurewerkz.welle.cache/basic-welle-cache-factory now has a 3-arity that accepts bucket name, content type and W to
use for cache writes.
clojurewerkz.welle.core/connect-to-cluster and clojurewerkz.welle.core/connect-to-cluster! are new functions that
use just like clojurewerkz.welle.core/connect and clojurewerkz.welle.core/connect! but use cluster clients.
Cluster client is a regular client (with exactly the same API) that does round-robin balancing of requests between multiple hosts in a cluster.
clojurewerkz.welle.core/connect-to-cluster-via-pb and clojurewerkz.welle.core/connect-to-cluster-via-pb! are the PBC
transport equivalents.
core.cache implementation no longer fails to compile when building uberjars.
clojurewerkz.welle.kv/delete-all-via-2i is a new convenience function that combines clojurewerkz.welle.kv/index-query
and clojurewerkz.welle.kv/delete-all: it concurrently deletes multiple keys retrieved via a 2i query:
(ns my.app
(:require [clojurewerkz.welle.kv :as kv]))
;; deletes multiple objects that have :stage index values between 10 and 20
(kv/delete-all-via-2i "pipeline" :stage [10 20])Documentation guides have been greatly improved.
ClojureWerkz Support dependency has been bumped to v0.4.0.
Welle now features a Ring session store implementation.
To use it, require clojurewerkz.welle.ring.session-store and use clojurewerkz.welle.ring.session-store/welle-store function like so:
(ns my.service
(:use clojurewerkz.welle.ring.session-store))
(let [store (welle-store "web_sessions")]
...)It is possible to pass :r, :w and :content-type options that will be used for reads and writes:
(ns my.service
(:use clojurewerkz.welle.ring.session-store))
(let [store (welle-store "web_sessions"
;; r
3
;; w
3
"application/json")]
...)By default, :w and :r of 2 will be used and :content-type is com.basho.riak.client.http.util.Constants/CTYPE_JSON_UTF8
clojurewerkz.welle.links namespace provides a DSL for Riak link walking operations:
(kv/store bucket-name "joe" {:name "Joe" :age 30} :content-type "application/clojure")
(kv/store bucket-name "peter" {:name "Joe" :age 32}
:content-type "application/clojure"
:links [{:bucket bucket-name :key "joe" :tag "friend"}])
;; this assumes you did (:use clojurewerkz.welle.links)
;; or equivalent in the current namespace
(walk
(start-at "people" "peter")
(step "people" "friend" true))clojurewerkz.welle.kv/store now takes the new :links option that lets you store
Riak links with the value. clojurewerkz.welle.kv/fetch then transparently deserializes when the value
is fetched back:
(kv/store bucket-name k v :content-type Constants/CTYPE_TEXT_UTF8 :links [{:bucket "pages" :key "clojurewerkz.org" :tag "links"}])
(let [fetched (kv/fetch-one bucket-name k)]
(println (:links fetched)))clojurewerkz.welle.buckets/update better reflects what the function really does. However, clojurewerkz.welle.buckets/create may
reflect the intent a bit better in certain cases so it is kept for backwards compatibility.
Initial map/reduce queries support has been implemented, everything related to it
resides under the new clojurewerkz.welle.mr namespace.
clojurewerkz.welle.cache provides an implementation of clojure.core.cache cache store
protocol on top of Riak.
clojurewerkz.welle.kv/fetch-one is a convenience function for fetching objects in cases where conflicts/siblings are not expected.
For example, it is common to use "last write wins" strategy for caches and such. clojurewerkz.welle.kv/fetch-one works
the same way clojurewerkz.welle.kv/fetch does (and accepts exactly the same arguments) but always returns a single object,
not a list.
In case the response contains siblings, a IllegalStateException will be thrown.
Validateur dependency has been upgraded to 1.1.0.
clojurewerkz.welle.core/connect now has one more arity that lets client id to be specified. In addition,
if client id is not specified explicitly, it will be generated and set.
clojurewerkz.welle.objects namespace was renamed to clojurewerkz.welle.kv
New application/json+gzip content type serializer allows Riak object values to be serialized as JSON and compressed
with gzip (using JDK's GZip implementation). Compatible with both HTTP and PB interfaces.
If content type passed to clojurewerkz.welle.objects/store is application/clojure, Clojure reader will be used to serialize
and deserialize object value. On Clojure 1.4+, this means you can transparently store and fetch objects that include dates, too
(thanks to 1.4's extensible reader/instant literal support).
Welle now works around this Java client bug that used to break
application/json; charset=UTF-8 serialization.
clojurewerkz.welle.objects/store now normalizes metadata by stringifying all keys and requiring that all values
are strings. This is due to the current (Riak 1.1) Java client limitations.
clojurewerkz.welle.objects/delete-all is a convenient way to delete multiple keys. Since Riak (as of version 1.1) does
not provide a way to delete multiple objects in a single request, this function will use clojure.core/pmap to perform
multiple concurrent deletion requests. This implementation may or may not be suitable for your use case.
Welle now supports 2i, both indexing and querying:
(wo/store "people" k v :indexes {:email #{"[email protected]" "[email protected]"});; returns a list of keys
(wo/index-query "people" :email "[email protected]")Welle now uses RawClient API of the underlying Riak Java driver. This makes the API a lot more Clojuric and much closer to Sumo. This also makes it more flexible, allowing us to perform automatic serialization/deserialization for stored objects for a few most commonly used content types and control conflicts resolution without heavy boilerplate Java interoperability code in end user applications.
Welle now uses Leiningen 2.