java


Huge number of objects, sharding and java EE


I'm new to java ee and I'm not sure how to implement a particular requirement.
I have need for a large set (millions) of objects that maintain a bunch of rules and state and present an API for clients. Each of these objects is long lived. Given that there're so many of these things, we'll likely need to shard them across many VMs and use RMI to access them.
My question is whether there's a Java EE approach to solve the problem of locating an instance of the object, allowing the clients to get a reference to the object without needing to be aware of which machine it is on.
I'm aware of JNDI, but I'm not sure that registering each of the objects in a JNDI directory is appropriate. Do I need to write a "Locator" library that can make itself aware of the VM that each object belongs to?
Without more specific details, let me venture forth several avenues for exploration.
If I'm reading you correctly, what you want is something akin to a DHT but for hosting and looking up objects (code+data) or service nodes, not just raw data. I'm not aware of any such platform, though it sure sounds like an interesting idea.
Java EE itself (as a spec) doesn't specify, nor does the reference implementation provide an "out of the box" solution for the massively distributed clustering & sharding I think you're looking for.
Glassfish (the Java EE RI) itself uses Shoal as a clustering framework, which can use either Grizzly or JGroups as the underlying group communications platform.
So—in your particular case, I would look into building out and upon JGroups for group communications. Then, instead of a central registry, we rely on DHT for service/object location. Look at how existing, successful DHT-based platforms (memcached, Apache Cassandra) implement partitioning & lookup, fault-tolerance and failover, and just adapt/adopt those. Then you can use RMI/RPC for client-server (service node) invocations.
Hope I'm making sense, and good luck! If you do roll this out yourself, see if you can open source it. ;)
I may not directly answer you question, but I know that Oracle Coherence can not just distribute data, but can also distribute calculation against that data.
Simple example code here.
You write your calculation through a class that implements com.tangosol.uti.InvocableMap.EntryProcessor. This will enable calculation
to take place at the server where the data exists.
One restiction is that the data needs to be serializable because it
moves through the network.
public class CalcLogic implements EntryProcessor {
....
//InvocableMap.Entry is the "data"
//You write your calculation in this process methods.
public Object process(InvocableMap.Entry entry {
(YourObjectType) obj = (YourObjectType)entry.getValue();
//do some calculation against obj here
entry.setValue(obj);
return null;
}
....
}

Related Links

How to determine a swing component?
printing InfModel earlier changes printed representation later?
Any reason to use JBoss 5 over GlassFish v2/v3 Prelude?
Running Apache Geronimo in production
Listing the files in a directory of the current JAR file
Garbage Collecting objects which keep track of their own instances in an internal Map
LINQ for Java tool [closed]
EJB vs Spring + POJO
Scala equivalent of Java java.lang.Class<T> Object
Class inheritance with Hibernate and hbm2java
Too Little CPU Utilization in Java
java decimal String format
JFileChooser for Python?
Javadoc-like Documentation for C++
Java Application Installers [closed]
Serialization of objects: no thread state can be involved, right?

Categories

HOME
pandas
google-chrome-extension
asp.net-core
openmp
heroku
react-virtualized
xmpp
packages
jxls
datastax-java-driver
bs4
node-notifier
quicklook
correlation
windows-phone-7
iron-router
quartz-scheduler
uitypeeditor
dosgi
opentracing
openrefine
windows-7-x64
lldb
clickonce
scriptcs
opencover
mps
vlsi
pingfederate
strncpy
vision
bpel
gsoap
stormpath
directx-10
galsim
create-table
google-sites-2016
grails-3.1
form-data
wallpaper
hexo
automake
greenrobot-eventbus
graphenedb
g-code
ibpy
mime
outlook-api
eclipse-gef
angular-resource
jquery-validate
lowpass-filter
nstextview
namecoin
or-tools
objective-c-swift-bridge
url-pattern
database-optimization
skobbler-maps
independentsoft
ios8-today-widget
lemon
retina
jqgrid-formatter
kcachegrind
interrupted-exception
fluid-dynamics
c++03
rake-task
operator-precedence
codeigniter-url
winddk
mdt
code-access-security
document-classification
rebol3
htmlcleaner
mysql-error-1062
mbr
prettify
dataadapter
wsdl-2.0
chuck
libstdc++
armcc
parametric-equations
appconkit
chrono
doh
xtype
digest-authentication
coda-slider
wise
filtered-index
bespin
data-driven
noscript
windows-live-messenger

Resources

Database Users
RDBMS discuss
Database Dev&Adm
javascript
java
csharp
php
android
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App