" /> testblog: April 2006 Archives

« December 2005 | Main | May 2006 »

April 30, 2006

OpenJMS Oracle database

....can be on the local machine, in fact probably should be, but wherever.

GPS-LAB7 is MS-SQL; GPSLAB8 is Oracle

I think that I have got it. Remote query results joined at client

Added "j" counter and runs through query response break to j > 1

This lets me process more than one message. Have run it twice in a row and it works.

Have to drop and create the temp tables first outside of JDBC!!!!

Caused by: java.sql.SQLException: [Microsoft][SQLServer 2000 Driver for JDBC][SQ
LServer]Cannot drop the table 'tbltemp1', because it does not exist in the syste
m catalog.

import java.util.regex.*; added to ddbmsConnection

If the query has no "where" clause

...the name of the global table is parsed out. That global table is then "iffed" to see if it is fragmented. If NOT, the look up the machine of the table, construct a message with that machine as a destination. If the global table is fragmented, get the names of the two fragments. In turn, drop and insert into temp tables the results of the local queries. (Where I am now--I have created one of the temp tables.) Join the two temp tables and send the results to dump on the client screen. Note that I don't have the logic in there yet to just pass the query if the global table is not fragmented.

April 29, 2006

Got table dropped and created and values inserted

I am done for the evening.

Made my own JDBC connection in loaddb.createtable

createTable method in LocalDB

...modeled on executeQuery, but, like executeDML, void return type.

Put the localDB stuff in Ddbms; go_all tests OK

Where to join the tables?

The messaging is a nightmare that I have no more time to work with. I am trying to make the Client the spot that I can use to make tables and join them. So I am trying to make my client a client on steriods--it will have a connection to the LocalDB. I just added the LocalDB code to the Client that I think I need, ran the program, and it worked.

Wait. But DdbmsConnection is receiving the aMsg, and it is where the contents of the message are getting dumped. So I will put the LocalDb stuff there instead.

To the client and figuring out how to dump the query result into a table.....

....actually, to DdbmsConnect and waitForQryResponseMsg.

Added "if query equals " to LocalDB to add header info for MLW calendar

OK, get the tblnames into string variables; now need to get rid of []

Keep forgetting the () at the end of toString!

executeDave in DBWorker to call executeQuery directly

To get around the one row problem, I added a method to DBWorker that calls the LocalDB executeQuery directly with a supplied, "real" SQL string, since I had one laying around anyway.

Now I can process the returned list to get the two values, the table names.

Having trouble with the getOneRowQueryValue

...because even by my limited rules, a global table is going to have two fragments, and that means there'll be two rows in the result. Grrrr. Thinking of top 1, something like that.

tblNode HubID column was 1 2 3 4 5 instead of 11111

Database script and if for frag in exemgr; now won't run; reverting

Create script to make Calendar fragments on Oracle

And then I will need to have one of these tables on the 07 server Oracle and the other on the 08 server as Oracle. For now, they both go on the same Oracle. I will include the scripts with the Oracle script builds.

How is the table fragmented? Horizontal or Vertical?

tblDictionary_Table Global_Table_Is_Fragmented
0 not fragmented
1 Horizontal
2 Vertical

Second global query--is the global table fragmented and how?

April 28, 2006

Got the column labels with the dump of the columns

Had to hard-code in the LocalDB array read of values the query and then added the text of the columns to the begining of the list. Ouch. But it works. Now have to read the results into a temp table, get a second query through, and do the join. Ouch, ouch.

Still getting (or getting again) the rollback JDBC message.

So what if every call to LocalDB sends back column names?

Added rsmd.getColumnName to LocalDB, but it returns hub info and hangs AP

How to add the column headings to the message?

This leads me to the DP code else if (aMsg instanceof MsgLocQry) clause. Which leads to localDB.executeQuery. executeQuery is of type List. So, if I want to append the column names to that list, it would be nice to be able to put them at the beginning.......

dumpQryRespMsg(aMsg) in DdbmsConnection is replaced with....

//dumpQryRespMsg(aMsg);
String str = aMsg.getOutputDumpString();
String[] words = str.split ("\n");
for (int i=0; i < words.length; i++)
System.out.println (words[i]);

The action moves to DdbmsConnection

The qryResponseMessage is getting dumped to the Client screen by the DdbmsConnection.java. So that is where I think that I want to send query results and do my join.

It is simplistic and it is ugly, but....

Ok. I have the client load the query form the client properties file. This is the global query. When the global query gets to the ExeMgr, I parse the query enough to dump the global table name into a variable. I test the variable. In my case so far, I do only one "if"--TBLSERVER--and if true, convert the global query string to the local query string. I think roughly speaking I am meeting the "lookup in GDD" to find the logical name of the machine that holds the table fragment. And the answer to th global query is returned to the client and dumped there.

Now--to put the results of the query into a temp table
Do a second query, from different machine; dump results to a table; and do a join.

Challenge: quantify the results, as in know the column headers; getting the results into a table.

After some other investigations, regex still seems best, but...

I haven't gotten it to work yet. I met with the professor yesterday, and, based on my explanation of where I was, it seems like I am doing pretty well. After today, back to the grind--I start my new position on May 1, Monday. So I won't bring my laptop to work anymore.

Noted that I have to cycle the Oracle Listener when I bring my laptop to work.

April 27, 2006

Java REGEX

I looked at the recommened ZQL, but don't know how long it'd take me to get that working. It is a class that parses SQL statements. I am looking into using the Java REGEX stuff instead. That looks promising.

And this morning, got the query to work with my config

The properties files, the batch files, and the GDD tables are set up well enough that I can pass a query into the Client as a property and have the result returned to the Client. The query is against a table fragment, however. Now I need to add logic to the AP ExeMgr to replace a global table with the correct fragment.

April 26, 2006

Late at night

I now have my DPs and my AP waiting for messages--much mucking around with config and batch files. (And to my chatroom.) Next is to get the client to get query results from a table fragment.

Finally got the global query to work

Sometime early this afternoon. Made me fee like if I smoked I'd've gone out and had a cigarette. It is crude, of course. I assume that since all queries come from the client, they get sent back there, too. And my query parsing is very crude, not allowing for any where clause. I think that will be the frosting on the cake.

Just now I finished up going back through my tables and making some adjustments to the scripts. Now I am going to try and convert the names of the DPs, AP, and tables in the configuration and java files.

Changed the names of the nodes and the chatroom in the CS532_Example_Oracle_rmi_jms_full file.

I have a an Oracle and an MS SQL in OR, but only the DP for the Oracle.

Stringbuffer answer

Stringbuffer is used by getOutputDumpString, but I should be able ot print the results with System.out.println. Delving into messages more.

XP SP2

Upgraded laptop to SP2 last night. Thought what the heck, might as well do it. First problem was that went to screen saver and couldn't log back in. There were two programs running. So waited. 11:00 and there were four programs running. 2 AM and still four. So I rebooted. A message flashed by about a DLL. Came back up, and I could log in and run my program. Now, at work, can't run program. Getting ORA-12518.

April 25, 2006

dumpMsg and ddbmsMsg

How does dumpMsg know that aMsg is of type ddbms? Ddbms.java has get)utputDumpString which is a StringBuffer, do have to figure out how to deal with that.

Msg interface looks like my boy

It has AdHocMessage, Dumpable. It has it all. I am now looking back at the DP code to see if I can have it do something with the message to route the query results back to the Client. Hmm, I will pull up the Client too.

AdHocMessage

This looks like it will allow me to control the destination of messages. Like, I want to send the query result directly to the Client? I don't like this. I would prefer if the DPs only talked to the AP, and the Client only talked to the AP.

MsgQryResult to bypass ReqProc?

So, maybe I need to put some code in the doGblQry --or-- as I call it, doQuery.

Queue message analysis, III.....

Actually, 2 things are going on: DP_TX does a ROLLBACK, and, seemingly
independently, AP_OK, which has output to console the query results sends an
error message to Client referencing Unexpected Query Result.


  1. AP dump goes to AppProc 189--ap.processRequests();

  2. to ReqProc 262 -- processMessage(ddbmsMsg);

  3. which leads to ReqProc 420 --

 // unexpected request results
   else if
(ddbmsMsg instanceof
MsgQryResult)
    {
    // If the
message is a Query result, something is confused
    //
this should have been caught inside the
doGblQry.
    throw new SampleException("Unexpected Query
Result, the AppProc is not processing this
Command");
       }

Query message analysis, continued


Now, it looks like everything is going good as:



  1. DP_TX receives the query string.

  2. Posts this message to the console: Local SQL   
    :select DNO, DName from Dept;

  3. Sends __QRY_RES__ to [AP_OK]

  4. Gets a __ROLLBACK__ message from AP_OK (note the absence of
    [braces].

My global query is based on the global insert. Maybe it is the case that I
need to intercept the ROLLBACK and do a commit or abort instead. Maybe an abort
is fine, since the tran is done and I don't need it anyway? It has delivered the
query result--to the AP, though it should go to the Client. For now, I will
settle on the code getting to the AP.

Realized that I'll have to reinstall SP2 for my laptop to work at UST.

Progress

DP_TX is getting a __LOC_QRY__ message, but is posting a response message that has the query in it, not the results.

Local SQL :select DNO, DName from Dept;
begin Posting message :__QRY_RES__
Posting to :[AP_OK]
end Posting message :__QRY_RES__

Cool, but still frustrating. Now I am sending around messages, so that is actually a goal accomplished.

But now onto fixing that they are the wrong mesagaes. Actually, one message is wrong. The AP displays:

1 SALES
2 ENGINEERING
3 MARKETING
5 Accounting

So, it seems like that message is supposed to be destined for the Client, but doesn't get there. I get this from the DP :

Rolling back JDBC
Rolled IT BACK!!!!
closing JDBC connection
IT IS CLOSED!!!
IT IS REMOVED FROM ACTIVE CONNECTIONS!!!
Aborted!

I am going to run the -commit scenario and see where the last insert message ends up.



-query (I have just edited the go_all.bat) pile-up:



  1. Client: Waiting for Begin Tx Response Message to console
  2. AP: __BEGIN_TRANSACTION__ to Client and non-dump
    __OK__ to Client.
  3. Client: Dump of start BEGIN-TX-response from AP which has
    the subject _OK_.
  4. Client, on console: select DNO, DName from
    Dept
    ;  followed by Waiting for QRY Response
    Message
    .
  5. AP gets __GBL_QRY__ from Client.
  6. AP starts processing the query. (Question: can I get a dump of the Request
    ID? --Checking. Hmm. Doesn't look promising; at least, not from the
    GlobalRequestID.java.)
  7. AP posting __LOC_QRY__ to [DP_TX].
  8. Immediately get "AP Process Messages ERROR: global query is not
    implemented"--but, it is commented out. I will try a build_all and start again.

Lists to strings and messages

I have now created a local query message and can send it to the node that has the specified table. Sort of. The message doesn't get to the destination because the destination looks like [[DP_TX]]. I have to get rid of the extra pair of braces.

Figuring that I have 7 things to do, and that each will take 2 days, and I am now on day 3 of number 1, I am a day behind.

April 24, 2006

REGEX (and Google) to the rescue

This regex code seems to have gotten me out of my jam:

void doQuery (String queryName, GlobalRequest gr) throws SampleException
{
String INPUT = queryName;
String REGEX = "\\b\\s+\\b";

Pattern p = Pattern.compile(REGEX);
Matcher m = p.matcher(INPUT);
StringBuffer sb = new StringBuffer();

while(m.find()){
String REPLACE = ",";
m.appendReplacement(sb,REPLACE);
}

m.appendTail(sb);
System.out.println(sb.toString());

}
This returns the query.

Now that I have the machine of the local table, now what?

What next?

Looks like yesterday was 16 hours (with big breaks.) Have to regroup. What is next?

April 23, 2006

Final of the evening

src/edu/stthomas/cs532sample/applicationProcessor/ExeMgr.java:237: executeQuery(
java.lang.String,java.lang.Object[]) in edu.stthomas.cs532sample.applicationProc
essor.DbWorker cannot be applied to (java.lang.String)
List result = DbWorker.executeQuery(sqlPropName);

Query of global table edited and ready to go--spaces

and chomping the semicolon after the table name. Crude.

Extracting the global table name

Here's the code so far for extracting the global table name.

public String removeSpaces(String s) {
StringTokenizer st = new StringTokenizer(s," ",false);
String t="";
while (st.hasMoreElements()) t += st.nextElement();
return t;
}
void doQuery(String queryName, GlobalRequest gr)
throws SampleException

{if (queryName == null)
{
throw new SampleException("Query Failed: Invalid Table");
}
String eQueryName = removeSpaces(queryName);
int pos = eQueryName.indexOf( "from" );
pos = pos + 4;
int length = eQueryName.length();

String sub = eQueryName.substring(pos, length);

Cross-linked IDs error means that it's time to restart OpenJMS

No carriage returns or new lines in properties files! Yuck. : (

Oh, there's all kinds of stuff in these ini files. So, I have decided that poking around with modifying the strings is outside of my scope. I am just going to make sure that the SQL statements in the properties files are clean. That also sort of answers the question that I had about why there were backslashes in the queries. They must have something to do with linebreaks. Turn \n into \\n, something like that.

Have to remember to build both Client and AP, because the queries are with the client.

Strings. This something that I know something about.

Got past the null query problem

Had to move the property loader call into the scenario_3 method. Now the query shows up all the way to the execution manager.

ReqProc, MsgGblCmd, MsgGblQry

The ReqProc is the bit in the previous post. Creates a variable from MsgGblCmd casting DdbmsMsg. So I am going to do that with the query in the same spot. Now, I have to compile the AP, not the client.

System.out.println(qryMsg.getSerializedContent()); at 353 in ReqProc returns null to the AP. Bummer.


Global query

if (ddbmsMsg instanceof MsgGblCmd)
{
MsgGblCmd cmdMsg = (MsgGblCmd)ddbmsMsg;

InsertUpdateInfo insInfo = (InsertUpdateInfo)cmdMsg.getSerializedContent();

exeManager.doInsert(insInfo.tableName, insInfo.colToValueMap, gr);
}
else if (ddbmsMsg instanceof MsgGblQry)
{
// processQry((MsgGblQry)ddbmsMsg, gr);
throw new SampleException("AP Process Messages ERROR: global query is not implemented");
}

The flow

Ddbms composes the message.
MyNode sends the message. MyNode.postMsg. Tries to see if the destination is the local machine.

public Node getMyNode()
{
return theNode;
}

The initial message gets posted to Hub, then I suppose that the AP uses the MyNode read to read the message.

There's some overloaded methods for you.

public Msg read(List subjectList)
public Msg read(String subjectString)

Good point-- all the messages go to the Hub. They don't go to the AP. Rather, they are addressed to the AP.

Post and Read are methods of Node, an interface in the jar file edu.stthomas.ddbms.comm.itf.

So what does the AP do with the message? Does it matter? Probably not. Need to go to the ExeManager and the Request Processor.

Following the global insert now:

ExeManager.doInsert calls databaseWorker.getOneRowQueryValue which asks for sqlGetGctType.

In the ExeMgr, there is this weird thing -- sqlGetGctType--

Oh. I think that I have been here before. All the blah-blah-blah about the queries of the global tables is listed in the AP propeties file.

The Node table has the node name (AP_OK, DP_TX, etc) and the hub ID and chatroom name. It is the Hub table that is only one row long.

GCT_Name Nod_ID
Proj 2001 DP_NY
Emp 2000 DP_MN
Dept 2002 DP_TX

Global query not implemented

The AP says:

edu.stthomas.cs532sample.shared.SampleException: AP Process Messages ERROR: glob
al query is not implemented
at edu.stthomas.cs532sample.applicationProcessor.ReqProc.processMessage(
ReqPro