Well Sam, doing scans sequentially is a problem because if there is a timeout, i must kill the process, so the subsequent scans get killed. Forking off separate processes is a good idea, but i have used lan scanners that use threads. I guess forking off new processes would be fine because i am just pinging the broadcast address and the routing software does the rest of the work. Also i am kinda interested to work on threads. A while back i tried to make a ip address based lan scanner, and i never got to finish it. Read up on threads later. Wanted to implement it. I am aware that it is generates bugs. But just wanted to try. On Thu, 2008-03-20 at 15:31 -0400, Sam Clemens wrote:

debayan wrote:

...

I am one of those guys trying to spread Linux in the college. I use managed to create a local repository (http://lug.nitdgp.ac.in/mukti/repository/ ) which has helped, but most of my friends do not want to switch to Linux because they would not want to miss the games they play on Windows such as CounterStrike and Age of Empires. So i began writing this program which scans the network every 1 minute or so for online game servers, and gives a popup alert in the taskbar and gives info such as number of players online, IP address of the server and the map that is on the server. Though quite simple to implement and somewhat lame, its surprising how effective it was in bringing new people to Linux. I want to do this as a GSOC project. Want to add as many games as possible. Will implement multiple scans using threads. I also envisage adding features such as chatting on the game server. Would also like to add a Gtk+ frontend. Skill required is C/C++ and sockets APIs. What do you guys think? Is it a nice idea, or do i need to scratch my head more? (I hope this mail is not too long)

Don't do use threading...either do the scans sequentially, or just fork off multiple stand-alone processes. You gain little by threading other than a ton of problems with from conditions and other very subtle bugs. -- Debayan Banerjee NIT Durgapur, West Bengal, India debayanin@gmail.com http://debayan.wordpress.com

-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Hmmmm, thats a guru talking. Yes you are correct. I was not aware of the poor windows context switching in specific. Forking off separate processes is fine. By the way, do you think the idea is good? Actually whether it is accepted or not, we(http://groups.google.com/group/lugsdu) shall work on it anyways. On Thu, 2008-03-20 at 15:57 -0400, Sam Clemens wrote:

debayan wrote:

...

Well Sam, doing scans sequentially is a problem because if there is a timeout, i must kill the process, so the subsequent scans get killed. Forking off separate processes is a good idea, but i have used lan scanners that use threads. I guess forking off new processes would be fine because i am just pinging the broadcast address and the routing software does the rest of the work. Also i am kinda interested to work on threads. A while back i tried to make a ip address based lan scanner, and i never got to finish it. Read up on threads later. Wanted to implement it. I am aware that it is generates bugs. But just wanted to try.

Threading is something you use when the operating system has extremely inefficient context switches (i.e. windows). There's very little need for threading in the Unix/Linux world.

However, if you want to share some information between the various processes, then grab some shared memory, and use that, or use some sockets, or even named pipes out in /tmp.

Basically, threading is a hack designed to overcome the grevious design flaws of the windows scheduler and context switcher. And, as I said before, debugging threaded code can be extremely difficult due to race conditions.

Just because a programming technique exists doesn't mean that it should be used for your particular problem, in the same way that you don't see anyone driving garbage trucks in a grand prix race.

...

On Thu, 2008-03-20 at 15:31 -0400, Sam Clemens wrote:

...

...

I am one of those guys trying to spread Linux in the college. I use managed to create a local repository (http://lug.nitdgp.ac.in/mukti/repository/ ) which has helped, but most of my friends do not want to switch to Linux because they would not want to miss the games they play on Windows such as CounterStrike and Age of Empires. So i began writing this program which scans the network every 1 minute or so for online game servers, and gives a popup alert in the taskbar and gives info such as number of players online, IP address of the server and the map that is on the server. Though quite simple to implement and somewhat lame, its surprising how effective it was in bringing new people to Linux. I want to do this as a GSOC project. Want to add as many games as possible. Will implement multiple scans using threads. I also envisage adding features such as chatting on the game server. Would also like to add a Gtk+ frontend. Skill required is C/C++ and sockets APIs. What do you guys think? Is it a nice idea, or do i need to scratch my head more? (I hope this mail is not too long) Don't do use threading...either do the scans sequentially, or just fork off multiple stand-alone processes. You gain

debayan wrote: little by threading other than a ton of problems with from conditions and other very subtle bugs.

-- Debayan Banerjee NIT Durgapur, West Bengal, India debayanin@gmail.com http://debayan.wordpress.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Will keep that in mind. Thanx a lot. On Thu, 2008-03-20 at 16:13 -0400, Sam Clemens wrote:

debayan wrote:

...

Hmmmm, thats a guru talking. Yes you are correct. I was not aware of the poor windows context switching in specific. Forking off separate processes is fine. By the way, do you think the idea is good? Actually whether it is accepted or not, we(http://groups.google.com/group/lugsdu) shall work on it anyways.

I think the overall project is a good idea. I just think that using threaded code for this particular problem is more trouble than it's worth.

If you REALLY want to work on threaded code...which is also appropriately threaded... there are sections of the kernel which are threaded....find an old computer that someone is getting rid of, and play around with modifying those sections of the kernel.

First rule of programming: Decide on your project... THEN decide on the programming techniques to use to implement the project.

In my college days, that's the strategy employed by my Electrical Engineering professors. In contrast, my Computer Science profs were generally.... lets just say they would teach a programming technique, and then assign projects for which the programming technique required was usually one of the worst ways to implement the project.

To choose your programming technique before even choosing a problem to solve is, as they say, "putting the cart in front of the horse."

...

On Thu, 2008-03-20 at 15:57 -0400, Sam Clemens wrote:

...

debayan wrote:

...

Well Sam, doing scans sequentially is a problem because if there is a timeout, i must kill the process, so the subsequent scans get killed. Forking off separate processes is a good idea, but i have used lan scanners that use threads. I guess forking off new processes would be fine because i am just pinging the broadcast address and the routing software does the rest of the work. Also i am kinda interested to work on threads. A while back i tried to make a ip address based lan scanner, and i never got to finish it. Read up on threads later. Wanted to implement it. I am aware that it is generates bugs. But just wanted to try. Threading is something you use when the operating system has extremely inefficient context switches (i.e. windows). There's very little need for threading in the Unix/Linux world.

However, if you want to share some information between the various processes, then grab some shared memory, and use that, or use some sockets, or even named pipes out in /tmp.

Basically, threading is a hack designed to overcome the grevious design flaws of the windows scheduler and context switcher. And, as I said before, debugging threaded code can be extremely difficult due to race conditions.

Just because a programming technique exists doesn't mean that it should be used for your particular problem, in the same way that you don't see anyone driving garbage trucks in a grand prix race.

...

On Thu, 2008-03-20 at 15:31 -0400, Sam Clemens wrote:

...

...

I am one of those guys trying to spread Linux in the college. I use managed to create a local repository (http://lug.nitdgp.ac.in/mukti/repository/ ) which has helped, but most of my friends do not want to switch to Linux because they would not want to miss the games they play on Windows such as CounterStrike and Age of Empires. So i began writing this program which scans the network every 1 minute or so for online game servers, and gives a popup alert in the taskbar and gives info such as number of players online, IP address of the server and the map that is on the server. Though quite simple to implement and somewhat lame, its surprising how effective it was in bringing new people to Linux. I want to do this as a GSOC project. Want to add as many games as possible. Will implement multiple scans using threads. I also envisage adding features such as chatting on the game server. Would also like to add a Gtk+ frontend. Skill required is C/C++ and sockets APIs. What do you guys think? Is it a nice idea, or do i need to scratch my head more? (I hope this mail is not too long) Don't do use threading...either do the scans sequentially, or just fork off multiple stand-alone processes. You gain

debayan wrote: little by threading other than a ton of problems with from conditions and other very subtle bugs.

-- Debayan Banerjee NIT Durgapur, West Bengal, India debayanin@gmail.com http://debayan.wordpress.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

No but even if you fork off separate processes, you get non-blocking IO programs using pipes.... why use threads just for that then?? On Thu, 2008-03-20 at 21:30 +0100, Anders Johansson wrote:

Sam Clemens wrote:

...

debayan wrote:

...

Well Sam, doing scans sequentially is a problem because if there is a timeout, i must kill the process, so the subsequent scans get killed. Forking off separate processes is a good idea, but i have used lan scanners that use threads. I guess forking off new processes would be fine because i am just pinging the broadcast address and the routing software does the rest of the work. Also i am kinda interested to work on threads. A while back i tried to make a ip address based lan scanner, and i never got to finish it. Read up on threads later. Wanted to implement it. I am aware that it is generates bugs. But just wanted to try.

Threading is something you use when the operating system has extremely inefficient context switches (i.e. windows). There's very little need for threading in the Unix/Linux world.

However, if you want to share some information between the various processes, then grab some shared memory, and use that, or use some sockets, or even named pipes out in /tmp.

Basically, threading is a hack designed to overcome the grevious design flaws of the windows scheduler and context switcher. And, as I said before, debugging threaded code can be extremely difficult due to race conditions.

Just because a programming technique exists doesn't mean that it should be used for your particular problem, in the same way that you don't see anyone driving garbage trucks in a grand prix race.

What on earth are you talking about?

Threading is very useful, and extremely ubiquitous, even more so in these days of multicore CPUs

But even on a single core, I find it very nice to know that my programs don't have to stop and wait for I/O to finish processing before moving on to the next task. Higher utilization is a good thing

Anders -- Debayan Banerjee NIT Durgapur, West Bengal, India debayanin@gmail.com http://debayan.wordpress.com

-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Anders Johansson wrote:

debayan wrote:

...

No but even if you fork off separate processes, you get non-blocking IO programs using pipes.... why use threads just for that then??

To avoid having to implement an event loop to check and see what happened to the IO?

you don't need that. The process blocks until the IO completes, then the scheduler puts the process back on the CPU.

To be able to use all your CPU cores?

Let the process scheduler handle that. It knows when a process has I/O ready to read or not.

Also, not all kernel activities are non-blocking

Anders

-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Ok please answer this, say i ping the broadcast address 10.0.255.255 (on my lan) on the terminal, how does the OS execute the ping? Does it fork off new processes or does it use threads? I am sure it is not sequential because it is pretty fast. Now if it is not sequential, i dont really need to use threads in my program. I just have to execute a broadcast ping. The switch takes care of the rest i think. On Thu, 2008-03-20 at 21:42 +0100, Anders Johansson wrote:

debayan wrote:

...

No but even if you fork off separate processes, you get non-blocking IO programs using pipes.... why use threads just for that then??

Wait, I just read what you actually wrote and not what I thought you wrote

Forking off multiple processes to do the scanning would be hideously ineffective compared to using threads. Unlike what "Sam" told you, threads are far more efficient than multiple processes on linux (on any operating system). There is a reason why threads are sometimes referred to as "light weight processes".

Also, what does "non-blocking IO using pipes" have to do with anything? I'm not even sure what it means. Are you talking about the select() call not blocking? It wasn't IPC I was talking about when I said that things blocked. Multithreaded programming is just a basic tool all programmers use to make the most of the hardware we have.

Some programs don't need it, but very many do. Almost all you see on your desktop uses multithreaded programming. Linux would suck on SMP if it didn't

Anders -- Debayan Banerjee NIT Durgapur, West Bengal, India debayanin@gmail.com http://debayan.wordpress.com

-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

That was a shocker to me.... by the way, i was going to print game server info by reading the packets they send back. The packets contain the info. I am sure there are apps written for achieving this, because it is a simple thing as i said in my first mail. I just think if this whole thing were a part of the openSUSE distro many windows gamers would spend more time on Linux. On Thu, 2008-03-20 at 21:54 +0100, Anders Johansson wrote:

debayan wrote:

...

Ok please answer this, say i ping the broadcast address 10.0.255.255 (on my lan) on the terminal, how does the OS execute the ping? Does it fork off new processes or does it use threads? I am sure it is not sequential because it is pretty fast.

Of course it's sequential. The program "ping" is started, it then repeatedly sends an ICMP echo request and then waits for a response (or a timeout). It is an extremely trivial program that doesn't involve processes or threads or anything

...

Now if it is not sequential, i dont really need to use threads in my program. I just have to execute a broadcast ping. The switch takes care of the rest i think.

Broadcast pings may be ignored, don't forget that. Many places do ignore them for security reasons

Besides which, you said you were also going to handle resources on those servers, not just see which machines on the LAN were alive, right? for the latter, you already have many other tools already written

Anders -- Debayan Banerjee NIT Durgapur, West Bengal, India debayanin@gmail.com http://debayan.wordpress.com

-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

On Thu, 2008-03-20 at 22:08 +0100, Anders Johansson wrote:

debayan wrote:

...

That was a shocker to me.... by the way, i was going to print game server info by reading the packets they send back.

Info in the packets?

Then I guess we're talking about something other than ping.

I guess you're talking about using some game server specific tool that connects and requests information. I would have no idea if that tool is multi threaded or not (especially since you haven't said which tool it is)

No, i am just broadcasting the packet that the game client broadcasts to find a server. I use a packet sniffer to see what each game sends in the packet. Its something like "\xff\xff\xff\xffTLsEngineQuery\xff" for CounterStrike 1.6. Upon receiving this packet the server responds with a 114 byte reply that contains info on the number of players online, the map currently being played etc..... so there is no tool involved.

But regular ping wouldn't give you any information you could use, other than that there is a network card with that IP address responding to pings

...

The packets contain the info. I am sure there are apps written for achieving this, because it is a simple thing as i said in my first mail. I just think if this whole thing were a part of the openSUSE distro many windows gamers would spend more time on Linux.

Honestly, I doubt it. Why would they want to run a whole OS just to get popups to tell them which machines they can connect to using their windows machines. Especially if it would involve either rebooting all the time, or running a complete separate machine

Well, like i said i used it in the college. The thing is, windows gamers would not want to come to Linux because they will miss a game. This app solves the problem by telling them when to switch to windows... or when to start the game they want.... whcih may well be on linux.

Most highend games already run their servers on linux, and most gamers swear by them. But that hasn't increased the linux desktop use among gamers

Anders

-- Debayan Banerjee NIT Durgapur, West Bengal, India debayanin@gmail.com http://debayan.wordpress.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Ok, that kinda answers all my questions. Thanx a lot, and sorry for being so confusing. Do you think it could be a gsoc project? On Thu, 2008-03-20 at 22:36 +0100, Anders Johansson wrote:

debayan wrote:

...

No, i am just broadcasting the packet that the game client broadcasts to find a server. I use a packet sniffer to see what each game sends in the packet.

That's not ping. And whether or not your program is multi threaded, that's something you'll have to tell us.

But now I think you meant if the actual operating system code that sends out your packets to the network is multi threaded. And the answer is yes, it is. Almost the entire kernel is multi threaded. This is why you have "locks" all over the place. You only need locks when you're doing multi threaded programming

But it's not threaded in the way you seem to imply. There isn't one thread per address you're talking to (and by the way, the broadcast address is one single address - the kernel doesn't break it up and send one packet to each 'real' address when you send to broadcast. That breakup gets done by the switch). It is multi threaded in the sense that the networking stack can work in parallel to other parts of the kernel and user space.

Anders -- Debayan Banerjee NIT Durgapur, West Bengal, India debayanin@gmail.com http://debayan.wordpress.com

-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

On Thursday 20 March 2008 13:47, debayan wrote:

Ok please answer this, say i ping the broadcast address 10.0.255.255 (on my lan) on the terminal, how does the OS execute the ping? Does it fork off new processes or does it use threads?

With very few exceptions, every command you issue to a shell causes the creation (via fork, the _only_ way to create a new process in Unix systems) of a new process to execute that command.

I am sure it is not sequential because it is pretty fast.

A simple program like "ping" is too simple to illustrate the use or advantage of thread-based programming models.

Now if it is not sequential, i dont really need to use threads in my program. I just have to execute a broadcast ping. The switch takes care of the rest i think.

If you look at all the hardware and all the software in a modern, general-purpose operating system (Linux, Windows, all the Unixes all qualify as such) there are massive amounts of parallelism across all the levels of its organization. Additionally, operating systems support concurrency of the processes they manage even in the absence of enough processors to run them all in parallel. Were it not for all the hardware parallelism and software concurrency, observed computer performance would be abysmal. Randall Schulz -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

On Thursday 20 March 2008 14:12, debayan wrote:

Randall, i meant does the OS fork off new processes/threads for each IP to be scanned? I know the shell forks off new processes. I am just asking about the working of the ping command, especially the broadcast part .... infact i will go through the source code just to be sure.

I don't believe the standard ping command can operate on multiple destination IPs at once. Nonetheless, if I were to write such a thing, I might use threads, but it would depend on how many concurrent pings I needed to support. If it were less than, say 20, I'd probably just use fork under the covers (of the CLI interface it specifies). But beyond that, it would be onerous to make the OS schedule so many full processes. But it's important to keep in mind that in modern operating systems the use of shared libraries makes the overhead of running many instances of a single program much less than it would be without such techniques. And in the case of a program with so little data to manage as a multi-address ping command, that would let you get away with just forking up to a moderate number of concurrently ping-ed addresses. Nonetheless, they're called "lightweight" (as Anders noted) for a reason. There's far less per-thread overhead than there is per-process.

Even a guru may be wrong!!

I used to be a guru. Now I can't even spell it.

...

-- Debayan Banerjee

Randall Schulz -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Greg Freemyer wrote:

On Thu, Mar 20, 2008 at 5:23 PM, Randall R Schulz wrote:

...

On Thursday 20 March 2008 14:12, debayan wrote:

...

Randall, i meant does the OS fork off new processes/threads for each IP to be scanned? I know the shell forks off new processes. I am just asking about the working of the ping command, especially the broadcast part .... infact i will go through the source code just to be sure.

I don't believe the standard ping command can operate on multiple destination IPs at once.

Randall,

New to me as well, but in my office network "ping -b 10.0.1.255" causes a lot of responses.

I would guess that ping has 2 threads. A sender and a receiver thread.

No, there are no threads, processes or signal handlers involved (actually there are signal handlers in use, but not for this purpose) It simply sends out an ICMP request, and then it looks in the OS buffer to see which requests have been received, and prints them out. If you get more requests than you have place for in the buffer, they will be lost Anders -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

It scans the lan as of now. Have not really thought of anything bigger yet. On Thu, 2008-03-20 at 20:53 +0100, Anders Johansson wrote:

debayan wrote:

...

So i began writing this program which scans the network every 1 minute or so for online game servers, and gives a popup alert in the taskbar and gives info such as number of players online, IP address of the server and the map that is on the server.

Out of curiosity, what exactly are you scanning? The local LAN? The entire internet? Registered game server hubs?

Anders -- Debayan Banerjee NIT Durgapur, West Bengal, India debayanin@gmail.com http://debayan.wordpress.com

-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Well, mine is a middle level university in India (say in the top 20 in the nation), where the lans are not as huge as other universities worldwide. We have around 500 computers online. On a busy gaming day, there are close to 20 game severs online. Games include Counterstrike, AOE, FIFA, Quake, UT and others. Other better colleges i am sure have more gamers. While wine still fights to become a good emulator, I wrote the program because it served the purpose in my college for the time being. Also, i am sure it could be implemented for the internet also. But havent thought about technological implications yet. I guess one could use libsoup. I used it once. On Thu, 2008-03-20 at 21:34 +0100, Anders Johansson wrote:

debayan wrote:

...

It scans the lan as of now. Have not really thought of anything bigger yet.

OK, well, do you suppose many people will have a use for it? most LANs aren't exactly crowded with game servers

It's an interesting idea for a medium sized LAN where you might find unexpected game servers, like, say a university, but I don't think a lot of other places will find a use for it

Anders -- Debayan Banerjee NIT Durgapur, West Bengal, India debayanin@gmail.com http://debayan.wordpress.com

-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org