Link to home
Start Free TrialLog in
Avatar of BeginToLearn
BeginToLearn

asked on

receiving

My code is unstable. Sometimes it works, sometimes it doesn't. I guess it depends on network speed or something else.

So please help to improve it. tks a lot.

If run debug, I can see some thing  weird:
   + BlockLength always  equa 0  after read( newsockfd,&BlockLength, sizeof(int))
   + result is alway equal 1 after read(newsockfd,buffer,BUFFER_SEND)

however, result equal 2048 after fwrite (buffer,1,BUFFER_SEND,fp);
 I need your explanation here.
  Server crashes because of strncpy().

client.c
server.c
Avatar of sarabande
sarabande
Flag of Luxembourg image

i don't know whether it is the only problem but the

   strcpy(buffer2,buffer);

is bad cause it requires a terminating zero char at end of buffer.

you only were reading BUFFER_SEND bytes from socket what is 2048 bytes. that doesn't include the '\0' you were needing for strcpy.

i still think it would be better you would send size of message (and perhaps message identifier) prior to sending the message.

then your receiver could read first the size of message:

     
int n;
     unsigned int size_to_read = 0;

     n = read(newsockfd, (char*)&size_to_read,sizeof(unsigned int));
    
     if (n != (int)sizeof(unsigned int))
         return error();

     unsigned int message_id = 0;

     n = read(newsockfd, (char*)&message_id,sizeof(unsigned int));
    
     if (n != (int)sizeof(unsigned int))
         return error();
    
     // if we were here we have total size of message and message id
     // first we check if message_id is the expected

    if (message_id != 3 || size_to_read == 0 || size_to_read >= BUFFER_SEND)   // assuming file 
         return error();

    unsigned int nread = 0;
    
    while (nread < size_to_read)
    {
          n = read(newsockfd, &buffer[nread], size_to_read - nread);
          if (n <= 0)
               return error();
          nread += n;
    }
    // we add the terminating zero here
    buffer[size_to_read] = '\0';

Open in new window



the last while loop is needed cause tcpip may not transfer the whole send buffer but only parts of it.

Sara  
Avatar of phoffric
phoffric

The client is sending a message (Total Number Of Files), a short message, and the server is trying to read BUFFER_SEND bytes (a long message). If there are timing differences such that the server wasn't running for awhile after the connection was accepted and if the client subsequently sent multiple messages, then the server code below might pick up part of the subsequent messages. And your code, as written, would lose part of those subsequent messages.

client:
void sendFileAllFiles ( SOCKET sock,  list<string>& myList)
{
        std::stringstream totalfile;

        cout<<" in sendFile myList size: "<<myList.size()<<endl;       
        totalfile << myList.size();
        string a = totalfile.str();
        char *tf = (char*)malloc(sizeof(char) *a.length() +1);
        memset(tf,'\0',sizeof(char)*a.length()+1);
        strcpy(tf,a.c_str());
        send(sock, tf,a.size(),0);

Open in new window

server:
int GetTotalFile (SOCKET sock)
{
        char buffer[BUFFER_LEN]="\0";
        int n;
	n = read(sock,buffer,BUFFER_SEND);

Open in new window

Avatar of BeginToLearn

ASKER

>>i still think it would be better you would send size of message (and perhaps message identifier) prior to sending the message.
>>then your receiver could read first the size of message:

Could you please elaborate more?


Yesterday phoffric and I talked about similar approach from Kdo, but I don't understand it well.

//  Send the file name

  BlockType = FILENAME;

  send (sock, &BlockType, sizeof (int), 0);
 
  BlockLength = strlen (Buffer);

  send (sock, &BlockType, sizeof (int), 0);  << why do we need to send BlockType second time?

  send (sock, Buffer, BlockLength, 0);

//  Send the data

  buffer= (char*)malloc(sizeof(char) * MAXBLOCK );

  while ((BlockLength = fread (Buffer, 1, MAXBLOCK, fp) > 0)
  {                
    BlockType = DATA;
    send (sock, &BlockType, sizeof (int), 0);
    send (sock, &BlockLength, sizeof (int), 0);
    send (sock, &BlockType, sizeof (int), 0);  <<<< why do we need to send BlockType second time?

    BytesSent + send (sock, buffer, result, 0);
  }
>> strcpy(buffer2,buffer); is bad cause it requires a terminating zero char at end of buffer.

    If you look at the client side, you see that what is being sent is a short filename. Since buffer (and buffer2) are zero'd out before the read, then the issue of a terminating null byte is not a problem here if the timing is good. Even if the timing is bad, if the client includes the terminating null byte of the filename, then there will be no problem with the strcpy.

   However, as mentioned in my above post, what is a potential timing problem is that if the server is delayed for awhile before getting to the read, then if the client subsequently sends multiple messages, then the server will get those subsequent messages in the same buffer that is used to simply get the filename. (You can see this if you learn how to look at memory in ddd and then stall the server at a breakpoint prior to the send.)

   In the event of a timing problem, then the strcpy gets the filename, but the next message that is in the buffer is currently thown away.

   Because of timing issues, you sometimes were able to successfully run your program, and sometimes not.

   So, the idea we were trying to stress in previous questions is that the server needs to know how many bytes are in the body of the message. The server can read first this body length number, and then read the exact length of the message.

   If you do this, you may wonder what happened to the subsequent  messages that were sent by the client. Those messages may partly be in the server's tcp stack, and partly in the client's tcp stack.

Why not just do things the way you think it should be, test it, and if you run into trouble, then we can help out. You can get ideas from us, but you should implement only what makes sense to you.
So, the problem seems to be with your having variable length messages. But the message header should be fixed length so that when the server is waiting on the next message, it can look for exactly the fixed length message header.

I recommend that your message header have at a minimum, the length of the body the rest of the message. If you want to add a message type, then for your application where the server is in lock-step with the client, that is optional, but nice. Since they are in lock-step, then if you include a message type, then the server has the additional task of verifying that the message type it receives is exactly the message type that it expects.
Please post any new comment or idea when you have it :)
       I will read all comments carefully after serveral hours because I am doing research on what to research about DITA plug in Eclipse. It's due tonight.
I suggest that you precede any variable length message (e.g., filename, filesize, or your new directory tree structure string) with a 32-bit integer that represents the size of the body of the message.
Or, if you prefer ASCII, then you precede any variable length message with a fixed length message that has the size of the body of the message that follows. This ASCII message should be padded with blanks. Either left or right alignment will work.
you mention about "precede"... I just wonder precede mean adding length message in front of the message  such as "messagelegth message" by using a special character to seperate them or meaning sending a message length then send message. please clarify. tks.    
 
As long as the message length for the body of the message is in a fixed length field, then it doesn't matter whether the client packs the message length and the body of the message into one send; or two sends, with the first send being fixed length, and the 2nd send being the body of the message.

If using one send, there is no need to have a special character to seperate them.

Whether the client uses one send or two, the server will always use two reads. The first being a fixed length read just to get the size; and the 2nd will be to get the body.
I will implement this idea embedding fixed message length precede message body. I favor for 1 send. and sever will use 2 reads . I believe make a fixed length field should be ok for me. I
Put a break on this client line, step through it, and see if you can identify and fix the problem:
>> while ((result= fread (buffer, 1, BUFFER_SEND, fp) > 0))
Also, in general, when using a library function that returns an error condition, then you always want to do thorough error checking. For example, see the Return Value of fread:
    http://www.cplusplus.com/reference/clibrary/cstdio/fread/

You should read the rules for every library function you use, and report error conditions (and probably exit in your program).
ASKER CERTIFIED SOLUTION
Avatar of sarabande
sarabande
Flag of Luxembourg image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Look at prototype of fread:
size_t fread ( void * ptr, size_t size, size_t count, FILE * stream );
   http://www.cplusplus.com/reference/clibrary/cstdio/fread/

I would modify the macros BUFFER_SEND and BUFFER_LEN to be changed to const size_t variables.
3 more hrs I gonna work this. if you have other idea, please post hihi. tks.
What problem are you having?
Not now. I gonna implement the fixed message length as you mentioned above once i get home. still at school now hix hix hix
I rewrite the code so everything will be in a fixed length. I have segmentation and fail to understand why.
Sending totalfile is ok. Send filename and its size has problem. Here is my observation:

at client line 118: i send the message . The message is ok bc it could print before sending
filenamensize is: /home/ubuntu/program/client 25870
just send file name and its size !!!!!

at server line 99, 96: it can't receive the message
Here is the message from client:       
File name and its size just received!!!. No content here
Segmentation fault

Please advise.
oh i forgot to upload files.
client.c
server.c
I took a quick look at client.c and immediately noticed:
        char filenamensize[BUFFER_LEN];
        filenamensize[BUFFER_LEN]= '\0';

and other similar problems.

I also noticed that you did not take advice about size_t in http:#35183475

Did I read it correctly that for the filename/filesize message, you are sending out a 2K buffer that has maybe < 100 chars in it? I would think that you would want to shorten it up a bit.
begintolearn,

phoffric is right. it makes no sense to send 2048 bytes with about 2000 spaces only to keep the message size constant.

if you would send the length of message before the message you would not have any problems with the variable sizes.

anyway you must rework your code and remove all the unnecessary statements.

Sara
I read about size_t fread ( void * ptr, size_t size, size_t count, FILE * stream ), i think it's ok to keep like we have it now.
Let me rework with sending the length of message before sending sending the message.
Today, I'll let Sara finish helping you. If there are still problems tonight (time permitting), I'll see if I can help. As I said, from your OP, you were only a few lines of code away from getting this to work. I have given hints, but I think it will benefit you greatly to try to learn how to isolate the problems. Your assignment does resemble some issues in industry; and there you don't get help too often. There are some other issues with your OP that I may have commented on in one of these questions. Although not directly related to the communications and timing issues that you are having, there is enough knowledge gained from that discussion to enable you to find out this other issue. Anyway, 3 days should be enough time to narrow the problem down once you get rid of the timing issues.

One way to handle all the advice is to write up a TODO list, and literally go through each one; and accept or reject. (But do let us know, so we don't keep harboring on a tip that you reject.) Oddly, the size_t tip I gave was a result of some unusual behavior that popped up recently but not earlier. I did start building slightly differently, and didn't need to dwell on the issue myself. But, in general, it is a good idea to be aware of different integer types and keep them consistent even if the -Wall option doesn't catch it.
I think i just finished the message1. Could you please confirm that i am on right track? If not, please let me know immediately because i want to be confidence before moving on next step.. tks for your patience.
client.c
server.c
i have finished all message1, message2, and message3. I run debugger to detect the error on server side due to to wrong size_to_read. I can't see what i did wrong. Please adivse. tks.

below is the copy of terminal

message1
total files need to receive are 6
n is 4and sizeof (insigned int) is 4
message 2
message 3
message 3
message 3
message 3
n is 4and sizeof (insigned int) is 4
message 2
get wrong size to read or message id: Success
ubuntu@ubuntu:~/program$

client.c
server.c
after i run debugger and print hardcopy of code, i see something wrong here.
   At message3, server side: size_to_read is 4;
                          client side: size _to_read is 0;

I can't find what is wrong.
and each time i see different wrong info !!!!clueless
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I forgot, in functions 1 and 2, include debug printf to indicate the total number of files being sent, and also the filename (and similar items in the server side).

In function 3, include debug printf to indicate the total number of bytes being sent for each individual chunks, and show similar in the server side.

The idea is to find the first mismatch and focus on that.
Let me rework by reorganize code in function asu sugegst
I just took a better look at what you are doing with the message id. For now I recommend commenting out all code related to message id. You are not handling it the way it is normally done. I'm not against using it, but since the addition is only adding problems, I suggest removing it (at least temporarily). Then to add it back in after your program is working, you can ask about the proper way to handle Message Headers.
Looked more.. I think you should make a backup (if you want) and just remove all references to message id. If you want to add it back in, you can learn how to do implement it correctly.
But am I on the right track ?
In your client:
     while ((result= fread (buffer, 1, BUFFER_SEND, fp) > 0))
add debug to print out result.
>> But am I on the right track ?
I think I mentioned in an earlier release (it may have been this question, or the previous question) that you were only a few lines of code away from success.

With the latest release, you need to fix more lines of code. All I'm suggesting is that you modularize your code to form a clear lock-step with the client. Then you should be able to better figure out what is going wrong.
Ic.i have prinfed hardcopy.let me take a break then reorgnize in module
If you can find the first occurence where I said you were just a few lines of code from having a working program, then if you modularize from that starting point, you might be better off.
About a few line... Comment,u wrote in this question hiho
I'll be offline till tomorrow. Just to be clear about the 3 message functions.. A single message now includes the length of the body of the message (in bytes) and the body of the message. You should not treat the length of the body of the message as a separate message.


I noticed this in client.c (which isn't wrong, but less is more) at line 122:
>> char * buffer ;
>> buffer= (char*)malloc(sizeof(char) *BUFFER_LEN);

One purpose of dynamic allocation is to deal with the case where you do not know the size of the buffer. In this case you are always creating a buffer of the same length. So, you can replace the two lines with:
   char buffer[ BUFFER ];
and remove the free(buffer).
good night. I am reworking now. see you tomorrow. tks a lot.
I just finished reworking client.c in a better module partition. Let me upload it so you can take a look. I am going to work on server.c right after I wake up. tks .
client.c
you do in sendOneFile

    unsigned int size_to_read = sizeof(buffer);

but in case it is the last trunk of the file to send, the size_to_read must be the remainder which you can get from the return value of the fread.

your program improves but you could still make it much shorter by only have one single sendMessage function which takes message id and (string) message as arguments and then does three send calls, one for the size_of_message (deduced from string length), one for the message_id and one for the buffer (textdata)  to send. the buffer could be 2k fixed char array where 2k is the maximum of all possible message sizes.

Sara
let me update :
   unsigned int size_to_read = sizeof(buffer); in sendOneFile
and sendMessage before working on server.c. tks

I just finished the update. Could you please take a look at it? I want to make sure it's good enough before i move to server.c work.
client.c
the sendMessage looks quite good, beside that i would use a fixed sized buffer:

   char pch[BUFFER_LEN] = { '\0' };

which you don't need to allocate (and don't need to free).

the BUFFER_LEN should be the max. size a message could have and you should check that it is the case:

   
unsigned int size_to_read= a.length();
   // would throw an assertion in debug mode if message is out-of-size
   assert (size_to_read > BUFFER_LEN);

Open in new window


I'll check the rest of client.c later.

Sara




sorry. the condition passed to assert must do a positive test.

  assert(size_to_read > 0 && size_to_read <= BUFFER_LEN);

Sara
just update client.c

I am working on server.c now.
client.c
in sendOneFile instead of

       
char buffer[BUFFER_LEN] ; 
        memset(buffer, '\0',BUFFER_LEN-1);

Open in new window


do

   
char buffer[BUFFER_LEN] = { '\0`};

Open in new window


that is much simpler and makes the buffer all zero. (by the way, the BUFFER_LEN-1 in the memset doesn't make any sense when the size of the buffer is BUFFER_LEN. you alway need to initialize all elements of the buffer. if you have to initialize an array (not a pointer) with memset you can do

      memset(buffer, '\0',sizeof(buffer)*sizeof(buffer[1]));  

what is correct for a arrays of any type.

Sara
here it is.
client.c
CLIENT:

I guess you uploaded the wrong version. Your client.c in http:#35199280 has errors and warnings (using -Wall).

But at top-level..

If you have a large buffer for reading in the trunk, and then you convert it to a string, and then you malloc a space to convert the string back to a char array, that is very inefficient for the large file transfers.

Missing function:  SendFileName which will call SendMessage. It can be called by SendOneFile, if desired.

Missing error handling in library calls.

Probably not enough debug statements; but you can add them if you find problems.
Looks like there's been a bit of activity between http:#35199280 and my post. Since Sara is actively involved, I'll let Sara handle the remainder of this question - it will be a cleaner post that way.

If you have other specific questions and ask them separately, then you should be able to get quick answers.
the following code is wrong

       
if( result == BUFFER_SEND)
        {
            size_to_read= sizeof(buffer);	
                }
        else
        {
            size_to_read= sizeof( result);
        }
        
        t (buffer);
            sendMessage( sock, message_id,t);
               
                memset(buffer, '\0',BUFFER_LEN-1);// reset buffer after sending
        }

Open in new window


and should be corrected.

the result is the number of bytes you have to send in any case.

so you always can use it for 'size_to_read' ...

     unsigned int size_to_read = result;

and there is no need for the if/else.

then the

   t (buffer);

probably doesn't compile.

you better do

  A)  t = string(buffer, result);

that would create a std::string with exactly the length you have read from file. you also could do

 B) t = buffer;

but differently to A that requires the buffer to be correctly terminated. so if using A you are less error-prone.

finally the memset should make all elements of buffer zero and not only BUFFER_LEN-1 elements.

Sara
I just wan to focus client.c on this question . Later I will open another question regarding to server.c. update :)
client.c
sendMessage and sendOneFile look good.

two minor things:

change to <  in

   assert (size_to_read >0 && size_to_read < BUFFER_LEN);

cause the BUFFER_LEN is 2048+1 and the size_to_read maximum is 2048.

change to buffer[0] in

  memset(buffer, '\0',sizeof(buffer)*sizeof(buffer[0]))

both things were my fault. sorry.

Sara
note, buffer[0] and buffer[1] have same size, so it doesn't matter for your case, but in general we also could have arrays with one element and than the buffer[1] is out-of-bounds (even that wouldn't crash and work but is simply not good code).

Sara
buffer[0] and buffer[1] have same size. but tks to make it logical. Updated.
I know server is reverse of client. but i need to print out client and rework server.
client.c
the last client looks good. can you build and run it?

Sara
Oh. I am working on server now.but I off line for 7 hrs