+ Reply to Thread
Results 1 to 14 of 14

Thread: Programming help     submit to reddit submit to twitter

  1. #1
    CoP Dynamis
    Join Date
    Oct 2006
    Posts
    279
    BG Level
    4

    Programming help

    My work is having a problem with people assigning jobs to our solvers and not entering the right amount of tokens to run the job. Once the jobs are ready to run, the server kills the job right away because there isn't enough tokens to run the job.

    I'm looking to write a program that would scan the output deck file of a program to search for key words and to assign them a value so that they could be tallied and the user would know how many tokens they would need to run a certain job.

    Where would be the best place to start? I haven't wrote anything in a very long time but I catch on pretty quick. What would be the best language to use for this?

    Thanks in advance.

  2. #2
    Salvage Bans
    Join Date
    Feb 2007
    Posts
    811
    BG Level
    5
    FFXIV Character
    Orinthia Warsong
    FFXIV Server
    Excalibur
    FFXI Server
    Bahamut

    Thinking in Java and assuming input is in a string, it sounds like you could use string's "split" method to break apart your input into an array of all the individual words (you set the delimiter, like a space or comma or something, as a regular expression). There's also "tokenizer" which does a similar thing and has its own sets of methods apart from a simple string split. There's a lot of info on usage online and most languages should have something similar.

    Once the string is split apart, you can then search through the results for exact matching keywords while keeping a tally value. Have your keywords stored in a simple text file for future ease of use/change (just stick to a consistent delimiter). For values, maybe make everything lower case for easier matching (java's string has .toLowerCase() for this).

  3. #3
    CoP Dynamis
    Join Date
    Oct 2006
    Posts
    279
    BG Level
    4

    I was looking more for something very simple to use for other people. Something that you'd be able to input the path of the file and have the program scan the output deck and spit out what addons are going to be used and what the total value of tokens is needed for each of the addons.

  4. #4
    Salvage Bans
    Join Date
    Feb 2007
    Posts
    811
    BG Level
    5
    FFXIV Character
    Orinthia Warsong
    FFXIV Server
    Excalibur
    FFXI Server
    Bahamut

    I tried to write a thought process as this should be language independent. I linked java's methods because I've used it the most so far for academic things, but you don't need java to do this (it can do this though, but that's up to you).

    Gist of your task: Look up how to open a file and search through it for your token words in your favorite language. Make a button to open a file, start reading it upon opening, keep a tally of each token word matched to your known words (one tally per token type), multiply by their respective values, add them up, and output the final value in some text box. Maybe show a list of the tallies to give you something to verify by hand counting while testing to ensure accuracy. Nothing fancy; one button and one text box.

    You should be able to do this without too much trouble, even if it's been a while since you last wrote something. If no starting point, try python or vb.net; vb will probably be easier.

  5. #5
    Murder machine with a motor in her nose
    Join Date
    Apr 2007
    Posts
    368
    BG Level
    4
    FFXI Server
    Carbuncle

    What's the file format? (aka how much work is it to find the tokens?)

    It seem like this could be written in 30 minutes if the input file is pretty simple.

  6. #6
    Smells like Onions
    Join Date
    Nov 2011
    Posts
    5
    BG Level
    0

    Here is a start using java. Can add in additional logic to assign values to the extracted value.

    Code:
    import java.io.*;
    import java.util.regex.*;
    
    public class Token {
    	public static void main(String[] args) {
    		String filename = "C:\\data.txt";
    		String line;    
    		String token;
    		
    		// Regex Pattern: leftBoundary(.*?)rightBoundary
    		Pattern p = Pattern.compile("\\[(.*?)\\]");
    		Matcher m;
    
    		try {
    			BufferedReader in = new BufferedReader(new FileReader(filename));
    	
    			while ((line = in.readLine()) != null) {
    				m = p.matcher(line);
    				if (m.find()) {
    					System.out.println(m.group(1));
    				}
    			}
    			in.close();
    		} catch(IOException e) {
    			System.out.println(e);
    			System.out.println("Unable to open file.");
    		}
    	}
    }

    Sample text inside data.txt:

    This is a sample text file.
    Token is always surrounded by brackets, such as [2].
    Use regular expression to extract token.

  7. #7
    Old Lady Harem of the House of Weave
    Join Date
    Jun 2007
    Posts
    589
    BG Level
    5
    FFXIV Character
    Mne Jing
    FFXIV Server
    Gilgamesh
    FFXI Server
    Cerberus
    WoW Realm
    Hyjal

    I think you guys are confusing tokens here. Not 100% sure he was talking about counting tokens in a string.... String Tokenizer might have a use here, but not in the sense that you're thinking. The file mentioned for scanning would be handy. That being said, this isn't a lengthy project based on original post.

  8. #8
    wotg torrent kitty :3
    Join Date
    Jun 2007
    Posts
    1,643
    BG Level
    6

    if it's really just open a file + count occurrence of x, then it's simply a question of presentation to choose the proper language. pretty much every programming language has the necessary methods for those tasks. choose whatever you're most experencied with and google your open/read file + regex methods.

    i personally would use python and google spit out this tutorial on first hit: http://palewire.com/posts/2008/04/07...ts-on-the-fly/

  9. #9
    CoP Dynamis
    Join Date
    Oct 2006
    Posts
    279
    BG Level
    4

    Sorry I guess I explained it wrong. It really is just opening a file and finding what words are in the output deck. Where the tokens come into play is for the licenses. Our compute node says that it takes 21,000 tokens to run the main program, the problem is that people use addons that they are not aware of and they cost additional tokens. Since they know the program itself takes 21,000 tokens, they assign the job 21,000 tokens to run when they put it in queue. Since they don't realize they are running addition addons with the program, the total cost to run the program might be 30,000 tokens and the compute node kills the job before it even runs.

    So if I can open the file (output deck) and have a program scan for say "bolt add-on", then the program will tell the user that they are using this add-on and will give them the additional cost of that add-on plus the original cost of the program.

  10. #10
    i'm awesome.
    Join Date
    May 2005
    Posts
    9,581
    BG Level
    8

    Learn to use grep and awk, any higher level language sounds like overkill for something so trivial. You could even write a shell script if you wanted this simple logic like 'if the phrase bolt add-on appeared then increment some token count' and display the result.

    # ezprgm.sh filename
    You need XXXX tokens to run this job on compute node XXXX.
    #

    Python, Java, etc. all fail because they assume those are installed which they may not be. Assuming you're running on a unix-based OS this will be super simple and portable.

    What is an output deck, a series of files, one file, a special file type? I am assuming it's just a single text file with normal encoding. You do not sound qualified because it seems like something more complex than you're making it out to be (tokens, shared computing possibly a supercomputer, some sort of job/token cost compilation).

  11. #11
    CoP Dynamis
    Join Date
    Oct 2006
    Posts
    279
    BG Level
    4

    Sorry i've been sick and haven't been around to responed. Ok so here's the deal.

    In my CAE group, we use a 3 compute node setup with a token system for the licenses. I'm not sure of the total amount of tokens we have for Nastram (the software we are having a problem with) but I think it's on the order of close to 100,000 tokens. The Output deck file is just as you said, a simple .dat file that has encoding in it.

    I'm not looking to do anything server side for this. All I want a workstation based application that requires no previous installtion of software that can run on Windows xp to Windows 7. Something super simple to scan the .dat file, show the users how many tokens would be required to run the job if they queued it instead of them having to guess on the tokens and having their job killed because they guessed wrong.

    For whatever reason, my boss picked me (I started 3 weeks ago) to start being in charge of how efficient we are and to utilize our off hours better when no one is around.

  12. #12
    Ridill
    Join Date
    Oct 2005
    Posts
    10,227
    BG Level
    9
    FFXI Server
    Asura

    Quote Originally Posted by Lost View Post
    Sorry i've been sick and haven't been around to responed. Ok so here's the deal.

    In my CAE group, we use a 3 compute node setup with a token system for the licenses. I'm not sure of the total amount of tokens we have for Nastram (the software we are having a problem with) but I think it's on the order of close to 100,000 tokens. The Output deck file is just as you said, a simple .dat file that has encoding in it.

    I'm not looking to do anything server side for this. All I want a workstation based application that requires no previous installtion of software that can run on Windows xp to Windows 7. Something super simple to scan the .dat file, show the users how many tokens would be required to run the job if they queued it instead of them having to guess on the tokens and having their job killed because they guessed wrong.

    For whatever reason, my boss picked me (I started 3 weeks ago) to start being in charge of how efficient we are and to utilize our off hours better when no one is around.
    What happens when you just put 100,000 tokens on each job? Will it only take what's necessary from it to perform the task?

    When a user inputs the number of tokens to use, is it saying "Run this job and you can use up to X tokens", or is it saying "Run this job and take X tokens, even if it's more or less than required"?

    It's stupid software design (on Nastram's end) to require you to tell it how many tokens to use when it already will know how many tokens to use, unless it's an "up to X tokens" type parameter, in which case just tell everyone to put 100000 in each time, and let it take what it needs.

  13. #13
    Relic Weapons
    Join Date
    Oct 2006
    Posts
    335
    BG Level
    4

    A job submitted to a computing cluster under license schemes like this will reserve the entire token/cost/hours estimate for its own job. This will prevent other users from being able to submit their jobs to the cluster (if you simply "estimate" the max number of available tokens), because there won't be a sufficient computing/licensing balance remaining for their jobs to run.

    It may seem stupid, but token-based licensing tends to be cheaper than per-seat and per-option licensing. Plus it's supposed to encourage better sharing of the cluster resources and prevent long compute-bound jobs that tie up licenses, etc.

    As for the OP, why not create a simple webpage that takes the contents of the output deck in a text box and process it with some Javascript after hitting a submit button? Even if you can't host it anywhere, you should be able to view it locally, and all of the workstations should have access to a modern internet browser. You should be able to make an exhaustive list of all the add-on keywords you need to find in the output deck's text and display the calculated cost in the window pretty easily.

  14. #14
    Ridill
    Join Date
    Oct 2005
    Posts
    10,227
    BG Level
    9
    FFXI Server
    Asura

    Quote Originally Posted by Teorem View Post
    A job submitted to a computing cluster under license schemes like this will reserve the entire token/cost/hours estimate for its own job. This will prevent other users from being able to submit their jobs to the cluster (if you simply "estimate" the max number of available tokens), because there won't be a sufficient computing/licensing balance remaining for their jobs to run.

    It may seem stupid, but token-based licensing tends to be cheaper than per-seat and per-option licensing. Plus it's supposed to encourage better sharing of the cluster resources and prevent long compute-bound jobs that tie up licenses, etc.
    Well, it's not the resource sharing method that's stupid, it's that you can't just tell it to take however many tokens it needs. It seems redundant to have to tell it how many tokens to use, when it already knows how many it needs. Why can't you just say "run this job" and it automatically takes the tokens it needs, and if there's not enough available in the pool (rather than relying on some number you type), it stops.

Similar Threads

  1. VB.Net Programming Help
    By Datonare in forum Tech
    Replies: 10
    Last Post: 2012-02-25, 00:17
  2. Java program Help: KI tracker
    By Skie in forum Tech
    Replies: 19
    Last Post: 2011-04-08, 09:31
  3. Programming help
    By Bloodfire in forum Tech
    Replies: 15
    Last Post: 2011-04-07, 16:40
  4. Beginner Programming Help Thread
    By Callisto in forum Tech
    Replies: 13
    Last Post: 2011-03-04, 18:24
  5. Replies: 8
    Last Post: 2010-02-20, 01:21
  6. Help finding a program
    By Darunia in forum Tech
    Replies: 4
    Last Post: 2007-07-10, 22:03