Loss is too high

[psinha30]

❓ Questions and Help

Hi, I was trying to train the mrcnn on a custom dataset but seems the loss is very high. Is this correct and also I dont see any signs of convergence.

the config file:
MODEL:
META_ARCHITECTURE: "GeneralizedRCNN"
WEIGHT: "catalog://ImageNetPretrained/MSRA/R-50"
ROTATED: True
BACKBONE:
CONV_BODY: "R-50-FPN"
RESNETS:
BACKBONE_OUT_CHANNELS: 256
RPN:
USE_FPN: True
ANCHOR_STRIDE: (4, 8, 16, 32, 64)
PRE_NMS_TOP_N_TRAIN: 2000
PRE_NMS_TOP_N_TEST: 1000
POST_NMS_TOP_N_TEST: 1000
FPN_POST_NMS_TOP_N_TEST: 1000

STRADDLE_THRESH: -1
ANCHOR_ANGLES: (-90, -60, -30)

BBOX_REG_WEIGHTS: (1.0, 1.0, 1.0, 1.0, 1.0)

ROI_HEADS:
USE_FPN: True

# weights on (dx, dy, dw, dh, dtheta) for normalizing rotated rect regression targets
BBOX_REG_WEIGHTS: (10.0, 10.0, 5.0, 5.0, 1.0)

USE_SOFT_NMS: True
SOFT_NMS:
  METHOD: 1

ROI_BOX_HEAD:
POOLER_RESOLUTION: 7
POOLER_SCALES: (0.25, 0.125, 0.0625, 0.03125)
POOLER_SAMPLING_RATIO: 2
FEATURE_EXTRACTOR: "FPN2MLPFeatureExtractor"
PREDICTOR: "FPNPredictor"
ROI_MASK_HEAD:
POOLER_SCALES: (0.25, 0.125, 0.0625, 0.03125)
FEATURE_EXTRACTOR: "MaskRCNNFPNFeatureExtractor"
PREDICTOR: "MaskRCNNC4Predictor"
POOLER_RESOLUTION: 14
POOLER_SAMPLING_RATIO: 2
RESOLUTION: 28
SHARE_BOX_FEATURE_EXTRACTOR: False
MASK_ON: True

MASKIOU_ON: True
ROI_MASKIOU_HEAD:
USE_NMS: True
DATASETS:
TRAIN: ("cocodataset_train","cocodataset_val")
TEST: ("cocodataset_val",)
DATALOADER:
SIZE_DIVISIBILITY: 32
SOLVER:
BASE_LR: 0.0005
WEIGHT_DECAY: 0.0001
STEPS: (60000, 80000)
MAX_ITER: 100000

OUTPUT_DIR: "checkpoints/rotated/mscoco_msrcnn"

[psinha30]

EXAMPLE TRAINING IMAGE