In the assembly of peg-hole composite components, traditional fixed-parameter admittance control struggles to adapt to varying contact conditions, leading to a high failure rate. To address this issue, this paper proposes a variable-parameter admittance control algorithm based on MADDPG with a segmented reward mechanism. This approach dynamically adjusts admittance parameters via reinforcement learning to enhance multi-robot collaborative assembly performance. Experiments were conducted on assemblies with three different tolerances (1mm, 2mm, and 3mm) and compared against fixed-parameter admittance control. Results demonstrate that the proposed method achieves rapid convergence across different tolerance levels and successfully completes the 2mm tolerance assembly within 100 steps, with a success rate exceeding 60%, significantly outperforming the fixed-parameter approach. The segmented reward mechanism effectively mitigates excessive contact forces and positional deviations, improving assembly stability. This research contributes to autonomous assembly in unstructured environments and has the potential to enhance robotic adaptability and reliability in industrial manufacturing.