How to run my personal cluster algorithm (java code) inside apache spark -


i have code other student it's k-medoids clustring algorithm, , should run inside apache spark, know spark based in rdd, algorithm complete , not know how change rdd's example part of code 

public queue<arraylist<object>> getfswoosh() {     queue<arraylist<object>> resultat = new linkedlist();     map<string, map<string, arraylist<object>>> p = new hashmap();     map<string, set<string>> n = new hashmap();      arraylist<object> courant = null;     arraylist<object> buddy;     arraylist<object> doc;     arraylist<object> fusion;     // preparation of sets "p" , "n"     (reglesimilarite regle : this.regles) {         string feature = regle.getsimilarites().get(0).getattribut();         (int = 1; < regle.getsimilarites().size(); i++) {             feature += "+" + regle.getsimilarites().get(i).getattribut();         }         p.put(feature, new hashmap());         n.put(feature, new hashset());     }       while (!liste.isempty() || courant != null) {         if (courant == null) {             courant = liste.remove();         }         buddy = null;         // new feature values registered (feature, value) -> null         (string feature : p.keyset()) {             //system.out.println("verification de nouveaux documents");             string valeur = pfv(feature, courant);             if (p.get(feature).get(valeur) == null) {                 p.get(feature).put(valeur, courant);                 //system.out.println("nouvelle valeur rencontrée");             }         }         // if value has been encountered (feature, value) -> (document! = current)         (string feature : p.keyset()) {             //system.out.println("verification de p s'il existe deja un document!=courant");             string valeur = pfv(feature, courant);             if (p.get(feature).get(valeur) != courant) {                 buddy = p.get(feature).get(valeur);                 //system.out.println("valeur trouvé dans p !!");             }         }         // if not find match in p, in '(result)         if (buddy == null) {             int index = 0; // indice de la regle similarite = ordre de la feature !? ( tester )             (string feature : p.keyset()) {                 //system.out.println("verification pour match "+feature+"=="+this.regles.get(index));                 string valeur_feature = pfv(feature, courant);                 if (!n.get(feature).contains(valeur_feature)) {                     if (!resultat.isempty()) {                         iterator<arraylist<object>> = resultat.iterator();                         while (it.hasnext()) {                             doc = it.next();                             if (this.matchfswoosh(doc, courant, index)) {                                 buddy = doc;                                 break;                             }                         }                     }                     if (buddy == null) {                         n.get(feature).add(valeur_feature);                     }                 }                 index++;             }         }         if (buddy == null) {             resultat.add(courant);             courant = null;         } else {             //system.out.println("fusion de "+courant+" "+buddy);             fusion = this.merge(courant, buddy);             resultat.remove(buddy);                (string feature : p.keyset()) {                 (string valeur : p.get(feature).keyset()) {                     if (p.get(feature).get(valeur) == courant || p.get(feature).get(valeur) == buddy) {                         p.get(feature).put(valeur, fusion);                     }                 }             }             courant = fusion;         }     }     return resultat; }

there way run code inside spark there, if not there way run complex code in rdds please if there other suggestion thank you


Comments

Post a Comment

Popular posts from this blog

php - Cannot override Laravel Spark authentication with own implementation -

during login process of laravel spark need request token of remote api. api stores exact same users , passwords laravel spark application. therefore need username , non hashed password of user during authentication process. i thought overriding authenticated method solution problem. in routes/web.php override post /login endpoint , pointing endpoint own logincontroller: <?php namespace app\http\controllers\auth; use illuminate\support\facades\log; use illuminate\http\request; use laravel\spark\http\controllers\auth\logincontroller sparklogincontroller; class logincontroller extends sparklogincontroller { /** * create new login controller instance. * * @return void */ public function __construct() { parent::__construct(); } /** * handle successful authentication attempt. * * @param request $request * @param \illuminate\contracts\auth\authenticatable $user * @return response */ public fu...
Read more

Qt QGraphicsScene is not accessable from QGraphicsView (on Qt 5.6.1) -

i new qt. trying catch mouse press , release in qgraphicsview area , draw line. this, extended qgraphicsview , called myqgraphicsview . seems scene not attached since added " hello world " , never appears in view. here code extended class: class myqgraphicsview : public qgraphicsview { q_object public: myqgraphicsview(qgraphicsscene *scene, qwidget *parent = q_nullptr) : qgraphicsview(scene,parent){ // qgraphicsview::qgraphicsview(scene,parent); } qpoint *pointptr, *startpos,*endpos; qgraphicsview *cv;//maa qgraphicsscene *c; qgraphicsscene *sceneptr; // maa protected: void paintevent(qpaintevent *event) { show(); } void mousepressevent(qmouseevent *event) { // pos wrt widget. global wer global screen qdebug()<<" mouse pressed @ "<<qwidget::mapfromglobal(event->globalpos()); //globalx(); startpos = new qpoint; startpos->setx(qwidget::mapfromglobal(event->globalpos()).x()); startpos->sety(qw...
Read more

What is happening when Matlab is starting a "parallel pool"? -

running parallel cpu processes in matlab starts command parpool() according documentation , function: [creates] special job on pool of workers, , [connects] matlab client parallel pool. this function takes bit of time execute, on order of 30 seconds. in other multi-cpu paradigms openmp, parallel execution seems totally transparent -- i've never noticed behavior analogous matlab (granted i'm not experienced parallel programming). so, happening between time parpool() called , when finishes executing? takes long? parallel computing toolbox enables run matlab code in parallel using several different paradigms (e.g. jobs , tasks, parfor , spmd , parfeval , batch processing), , run either locally (parallelised across cores in local machine) or remotely (parallelised across machines in cluster - either 1 own, or 1 in cloud). in of these cases, code run on matlab workers , copies of matlab without interactive desktop. if you're intending run on remo...
Read more