Relibility issues

[matahap]

Hello ,
I am doing a master thesis project and I am comparing(biological variables)between 3 groups (10 each) , under microscope , Do I have to have another observer with me ? or I am enough !

If that ok

I will proceed to do calibration with my supervisor ? but can any one explain to me briefly how it is done or give me a good reference .

after I will be calibrated I will do intra - rater reliability using Kappa ? but my data will be ordinal , so what is your statistical advice regarding the test because I read about "weighted kappa " what do you think ?

Please really I am here because I am in critical situation in my master and I am looking for your valuable advice

Thank you waiting for replies ........

[kingkong]

Weighted Kappa is the appropriate analysis for ordinal data. But there is no such thing of intra-rater reliability. There are two main types of reliability which relevant to experimental study. First, Inter-rater reliability means whether the results from both of you and another rater on the same subject are consistent among each other at one time. Second, test-retest reliability means the same subject is measured from one time to another time to see if the results are consistent at different times. I think you are talking about inter-rater reliability, then you need to find another rater to compare the same sample at the same time for you.