Skip to content

a simple, reliable and easy-to-use java config system base on taobao diamond

Notifications You must be signed in to change notification settings

bingoohuang/diamond-miner

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

83 Commits
 
 
 
 
 
 
 
 

Repository files navigation

diamond-miner

a java config system base on taobao diamond, main changes included:

  • API is simplified to be more friendly.
  • All files charset changed from GBK to UTF-8.
  • Logging system changed from commons logging to slf4j.
  • All code were cleaned and refactored.
  • Description and valid fields were added for convenience.
  • Diamond server disk files are rooted as use home directory.
  • A cache was added to client based on diamond and guava cache.
  • Some other changes.

Usage

  • Setup mysql database and tables
  • Startup diamond-server
  • Try to use diamond-client

Setup mysql database and tables

  • create database diamond
    create database diamond;
  • create table
-- for MySQL
create table diamond_stones (
    id varchar(255) not null,
    data_id varchar(255) not null,
    group_id varchar(128) not null,
    content longtext not null,
    description longtext default null,
    valid tinyint(1) not null default 1,
    gmt_create datetime not null,
    gmt_modified datetime not null,
    primary key (id),
    unique key uk_diamond_datagroup (data_id,group_id)
) default charset=utf8;
-- for ORACLE 
create table diamond_stones (
    id varchar2(400) not null primary key,
    data_id varchar2(255) not null,
    group_id varchar2(128) not null,
    content long not null,
    description varchar2(1024),
    valid number(1,0) default 1,
    gmt_create date not null,
    gmt_modified date not null
);

create index idx_diamond_stones on diamond_stones(data_id, group_id);
  • create user
-- for MySQL
create user 'diamond'@'%' identified by 'diamond';
grant all privileges on diamond.* to diamond@'%';

create user 'diamond'@'localhost' identified by 'diamond';
grant all privileges on diamond.* to diamond@'localhost';

Setup diamond-server

1. download the diamond-server code from github
2. run the command:
mvn package && cd target && java -jar diamond-server-0.0.8-exec.war -httpPort 17002
  • Cluster example
    1. java -jar diamond-server-0.0.8-exec.war -httpPort 18001
    2. java -jar diamond-server-0.0.8-exec.war -httpPort 18002
    3. Add config: dataId=nameservers, group=admin, content=localhost:18001 localhost:18002

The default mysql connection is:

db.driver=com.mysql.jdbc.Driver
db.url=jdbc:mysql://localhost:3306/diamond?useUnicode=true&&characterEncoding=UTF-8&connectTimeout=1000&autoReconnect=true
db.user=diamond
db.password=diamond
db.initialSize=10
db.maxActive=10
db.maxIdle=5
db.maxWait=5
db.poolPreparedStatements=true

If you have the different mysql ip, user or passoword, you can have several ways to change your connection info:

  • put a diamond-jdbc.properties in your current directory(same directory with diamond-server-0.0.X.war).
  • update WEB-INF/classes/diamond-jdbc.properties in diamond-server-0.0.X.war.
  • put a diamond-jdbc.properties in ~/.diamond-server/。

After you setup diamond-server successfully, you can open http://localhost:17002/diamond-server to login in diamond-server console. The default username and password is admin/admin.

Try to use diamond-client

  • Setup connection info for client
    • Create diamond-client.properties in the classpath root or in ~/.diamond-client/.
    • Add one line to diamond-client.properties
DiamondServer.address=localhost:17002

or cluster way

DiamondServer.address=localhost:18001 localhost:18002
  • Simple use examples
String foo = new Miner().getString("foo"); // foofoo
String bar = new Miner().getStone("my_group", "bar"); // barbar

int defaultTimeout = 10;
int timeout = new Miner().getInt("timeout", defaultTimeout);
  • Add a listener to repsonse config changing.
DiamondManager diamondManager = new DiamondManager("foo");
diamondManager.addDiamondListener(new DiamondListener() {
    @Override
    public Executor getExecutor() {
        return null;
    }

    @Override
    public void accept(DiamondStone diamondStone) {
        System.out.println("DiamondListener:" + diamondStone.getContent());
    }
});

String diamond = diamondManager.getDiamond();
  • Use substitution (Supported by DiamondMiner, and N/A in DiamondManager)
String foobar = DiamondMiner.getString("foobar");
// ${foo}-${bar} will substituted to foofoo-barbar
  • Use diamond cache
    1. Create a cache updater class, and it should implement Callable interface.

       package org.n3r.diamond.client;
      
       import org.n3r.diamond.client.cache.ParamsAppliable;
       import java.util.Arrays;
       import java.util.Date;
       import java.util.concurrent.Callable;
      
       public class DemoUpdater implements Callable<String>, ParamsAppliable {
           private String param;
      
           @Override
           public String call() {
               return param + new Date();
           }
      
           @Override
           public void applyParams(String[] params) {
               this.param = Arrays.toString(params);
           }
       }
      
    2. Add a config like:

       dataId=foo, group=DEFAULT_GROUP,
       [email protected]("Hello world") @TimestampMark("2013-10-09 10:37:08.123")
      
    3. Get the cache:

       String cache = DiamondMiner.getCache("foo");
      
    4. Update the cache manually, any change of config will cause cache updated.

       [email protected]("Hello world") @TimestampMark("2013-10-09 10:37:10.234")
       or
       [email protected]("你好,中国") @TimestampMark("2013-10-09 10:37:10.234")
      

Notes

  • garbled characters on TOMCAT console: add -Dfile.encoding=UTF-8 to VM options
  • check charset in MYSQL :
show create database diamond;
show create table diamond_stones;

set names utf8;
ALTER DATABASE `diamond` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `config_info` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;

Rantionale

  • The client and server local store tree:
    ~/.diamond-client              ~/.diamond-server
    |____config-data               |____config-dump
    |____DiamondServer.address     | |____admin
    |____snapshot                  | | |____users.diamond
    | |____DEFAULT_GROUP           | |____DEFAULT_GROUP
    | | |____foo.cache             | | |____bar.diamond
    | | |____foo.diamond           | | |____foo.diamond
    | | |____SOLR_URL.diamond      | | |____SOLR_URL.diamond
  • The client api will lookup config in privileged by order:
    • The local disaster recovery dir(~/.diamond-client/config-data)
    • Httpclient to diamond-server if local cache missed
    • Local snapshot

So if the diamond-server died, the client can still work by snapshot. And even when the snapshot is not available, the local disaster recovery dir can be used temporarily.

About

a simple, reliable and easy-to-use java config system base on taobao diamond

Resources

Stars

Watchers

Forks

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •