java


Huge number of objects, sharding and java EE


I'm new to java ee and I'm not sure how to implement a particular requirement.
I have need for a large set (millions) of objects that maintain a bunch of rules and state and present an API for clients. Each of these objects is long lived. Given that there're so many of these things, we'll likely need to shard them across many VMs and use RMI to access them.
My question is whether there's a Java EE approach to solve the problem of locating an instance of the object, allowing the clients to get a reference to the object without needing to be aware of which machine it is on.
I'm aware of JNDI, but I'm not sure that registering each of the objects in a JNDI directory is appropriate. Do I need to write a "Locator" library that can make itself aware of the VM that each object belongs to?
Without more specific details, let me venture forth several avenues for exploration.
If I'm reading you correctly, what you want is something akin to a DHT but for hosting and looking up objects (code+data) or service nodes, not just raw data. I'm not aware of any such platform, though it sure sounds like an interesting idea.
Java EE itself (as a spec) doesn't specify, nor does the reference implementation provide an "out of the box" solution for the massively distributed clustering & sharding I think you're looking for.
Glassfish (the Java EE RI) itself uses Shoal as a clustering framework, which can use either Grizzly or JGroups as the underlying group communications platform.
So—in your particular case, I would look into building out and upon JGroups for group communications. Then, instead of a central registry, we rely on DHT for service/object location. Look at how existing, successful DHT-based platforms (memcached, Apache Cassandra) implement partitioning & lookup, fault-tolerance and failover, and just adapt/adopt those. Then you can use RMI/RPC for client-server (service node) invocations.
Hope I'm making sense, and good luck! If you do roll this out yourself, see if you can open source it. ;)
I may not directly answer you question, but I know that Oracle Coherence can not just distribute data, but can also distribute calculation against that data.
Simple example code here.
You write your calculation through a class that implements com.tangosol.uti.InvocableMap.EntryProcessor. This will enable calculation
to take place at the server where the data exists.
One restiction is that the data needs to be serializable because it
moves through the network.
public class CalcLogic implements EntryProcessor {
....
//InvocableMap.Entry is the "data"
//You write your calculation in this process methods.
public Object process(InvocableMap.Entry entry {
(YourObjectType) obj = (YourObjectType)entry.getValue();
//do some calculation against obj here
entry.setValue(obj);
return null;
}
....
}

Related Links

How to save data in a database and open new activity after it is saved?
Incompatible classes of java generics
How to write a swap method that takes an array and two integers [duplicate]
Looking for solution to get a count of based on the Key in Java
Convert Image to base64 String in Java
Why doesn't the following piece of code input data properly?
Hibernate and HSQLDB - criteria sql restrictions for dates
How To Remove Item from Spinner using another spinner
Error while accessing network state
GCM integration with VPN
How to Decode values in listView/cursor/adapter(from Database) / modify every item in list in android ?
How to disable push notification from pushbots
Can I send the user to another .xml with only MAinActvity.java class?
How to send something from php page to java application?
disconnecting server from socket in chat program
Java arraycopy Method

Categories

HOME
pdf
blogger
activiti
json.net
plot
grep
android-4.4-kitkat
google-oauth
dxl
indesign
bs4
android-youtube-api
webrequest
adfs
gorm
kibana-4
fancybox-3
handsontable
modx-revolution
google-cloud-ml
timeout
vaadin7
systemc
telephony
angular-ui
footer
solaris-10
text-rendering
oxyplot
excel-vba-mac
google-pagespeed
custom-wordpress-pages
amazonsellercentral
zapier
autoconf
clickonce
excel-2007
bootstrap-tour
cultureinfo
restful-authentication
icloud-api
firebase-crash-reporting
android-widget
buildbot
cas
status
bootstrap-duallistbox
kannel
brunch
phpfox
fopen
spring-mybatis
bcd
http-status-code-503
functor
airconsole
calibre
kendo-ui-grid
catalog
jna
lumberjack
optix
android-tabhost
grid.mvc
nand2tetris
modelmapper
sencha-touch-2.3
flickr-api
rainbowtable
hibernate-tools
pubmed
pnotify
elgg
carrot
kbuild
memory-alignment
titanium-android
webdriverjs
angstrom-linux
actionbardrawertoggle
hill-climbing
ready-api
livequery
ford-fulkerson
lib.web.mvc
map-projections
appfabric-cache
apache-commons-fileupload
fluid-dynamics
system32
key-management
dia
operator-precedence
typekit
pick
xc16
socketexception
fluentautomation
jubula
jquery-layout
balanced-payments
geos
qt-faststart
distutils
frameset
netbeans-6.9
buildr
php-parser
invite
cinema-4d
disclosure
paster
gallio
preference
fixed-width
avatar
document-conversion

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App